Best Vision-Language Models to Run Locally in 2026

Rankings 2026-09-25 Last updated 2026-09-25 11 min read By Q4KM

Quick Answer

Qwen2.5-VL-7B-Instruct (Qwen) is the top pick as of September 2026 for local vision-language model workloads [9]. The local ranking places rank 1 Qwen2.5-VL-7B-Instruct (Qwen), rank 2 Qwen3-VL-8B-Instruct (Qwen), rank 3 Qwen3.5-4B (Qwen), rank 4 Qwen3.5-9B (Qwen), and rank 5 gemma-4-26B-A4B-it (google) [9][2][12][8][7].

Key Takeaways

How do these open-weight vision-language models compare on specs and benchmarks?

Model Org Params Quant/VRAM Context License Key benchmark (date)
Qwen/Qwen3.5-4B [12] Qwen [12] 4.7B parameters [12] BF16 weights 9.3 GB [12] 256K tokens [12] Apache-2.0 [12] not published [12]
Qwen/Qwen2.5-VL-7B-Instruct [9] Qwen [9] 8.3B parameters [9] BF16 weights 16.6 GB [9] 128K tokens [9] Apache-2.0 [9] MMMU_VAL 58.0 [11], MMBench_TEST_EN 83.2 [11], MathVista 68.1 [11] (as of 2025-09-17) [11]
Qwen/Qwen3-VL-8B-Instruct [2] Qwen [2] 8.8B parameters [2] BF16 weights 17.5 GB [2] 256K tokens [2] Apache-2.0 [2] not published [2]
Qwen/Qwen3.5-9B [8] Qwen [8] 9.7B parameters [8] BF16 weights 19.3 GB [8] 256K tokens [8] Apache-2.0 [8] not published [8]
google/gemma-4-26B-A4B-it [7] google [7] 25.8B parameters [7] BF16 weights 51.6 GB [7] 256K tokens [7] Apache-2.0 [7] not published [7]

Which open-weight vision-language models can run locally?

1. Qwen2.5-VL-7B-Instruct

Qwen2.5-VL-7B-Instruct from Qwen earns a place in the local list because it has an 8.3B parameter count, a 128K-token context length, and Apache-2.0 licensing [9]. Its Hugging Face download count was 6,530,663 over the last 30 days as of 2026-09-25 [9], showing broad adoption for local experimentation. The model was first published on Hugging Face on 2025-01-26 [9].

The BF16 weight footprint is 16.6 GB [9], which sets the minimum model-weight memory budget before context and runtime state. For general visual reasoning, OpenCompass OpenVLM data from 2025-09-17 lists MMMU_VAL 58.0, MMBench_TEST_EN 83.2, and MathVista 68.1 [11]. A practical local use is general multimodal question answering and visual reasoning tasks.

One caveat is that the OpenCompass scores are dated 2025-09-17 [11], while the model card was snapshotted on 2026-09-25 [9]; also, its 128K-token context length [9] trails the 256K-token contexts of several newer Qwen releases in this set [1][2][7][8]. The release date is listed on the model card [9].

2. Qwen3-VL-8B-Instruct

Qwen3-VL-8B-Instruct, from Qwen, ranks here as an open-weight vision-language model with 8.8 billion parameters, a 256K-token context window, and an Apache-2.0 license [2]. The model card lists BF16 weights of 17.5 GB, first publication on Hugging Face on 2025-10-11, and 19,149,628 downloads in the 30 days ending 2026-09-25 [2]. The repository name identifies the model as an instruct-tuned variant [2].

For local runs, the stated BF16 footprint is 17.5 GB, so a workstation GPU with at least that much free memory is needed before KV cache and activations are added [2]. The model card lists an Apache-2.0 license for the checkpoint [2]. A practical use is an on-premise multimodal assistant for screenshots, product images, or short technical documents where the 256K-token window can hold long context [2].

The main caveat is that the listed checkpoint reports BF16 weights of 17.5 GB, so smaller single-GPU boxes may need quantization or offloading to fit the runtime [2]. The model card links to the Qwen3 Technical Report, arXiv 2505.09388, dated 2025-05-14 [3].

3. Qwen3.5-4B

Qwen3.5-4B by Qwen is a compact published vision-language candidate for local deployment, with 4.7B parameters [12]. The model fits the shortlist because BF16 weights are 9.3 GB [12], context length is 256K tokens [12], and the license is Apache-2.0 [12]. First published on Hugging Face on 2026-02-27, Qwen3.5-4B recorded 7,029,433 downloads in the last 30 days as of 2026-09-25 [12].

Local hardware should provide 9.3 GB for BF16 weights [12] plus working memory for activations, batching, and context. Such a footprint suits desktop GPUs, unified-memory computers, or low-power edge systems when a modest vision-language assistant is needed. The model suits image questions, short document review, and on-device multimodal chat where 256K-token context [12] supports extended sessions.

A practical caveat is that constrained systems need headroom beyond the 9.3 GB BF16 figure [12] for context, activations, batching, and runtime state. Teams should validate outputs on target screenshots, receipts, charts, and real-world local documents before relying on it in local production workflows.

4. Qwen3.5-9B

Qwen3.5-9B by Qwen is a compact Apache-2.0 [8] candidate for local vision-language use because it combines 9.7B parameters [8], a 256K-token context [8], and a BF16 checkpoint of 19.3 GB [8]. The repository recorded 9,781,405 downloads [8] in the last 30 days to 2026-09-25 [8], supporting its placement among practical local models published on Hugging Face on 2026-02-27 [8].

For BF16 inference, plan for at least the 19.3 GB [8] checkpoint size in GPU memory, plus additional VRAM for activations and long prompts. That allocation fits a high-memory workstation GPU, making the model useful for private document review, screenshot analysis, or multi-frame visual questions where data should stay on local hardware and workflows remain inside controlled infrastructure.

The permissive Apache-2.0 [8] license and long 256K-token [8] context suit repeated internal workflows, such as extracting text from pages or comparing images across a session. A caveat is that benchmark validation on your own images, documents, and charts remains necessary before production in regulated, audited, or high-risk settings.

5. gemma-4-26B-A4B-it

gemma-4-26B-A4B-it by google ranks here because its 25.8B parameter Apache-2.0 package ships with a 256K-token context and BF16 weights of 51.6 GB [7]. The repository had 11,073,424 downloads in the last 30 days as of 2026-09-25, and it first appeared on Hugging Face on 2026-03-11 [7]. The Gemma 4 Technical Report documents the family on Hugging Face papers [6].

For local deployment, the checkpoint requires memory large enough for 51.6 GB of BF16 weights before activations and context state [7]. A workstation with high-capacity GPU or accelerator memory can host that footprint; a low-memory laptop cannot. The 256K-token context suits long-context tasks that need extended reading windows [7].

A practical use is an Apache-2.0, long-context workflow where license terms matter and the host has enough memory for 51.6 GB BF16 weights [7]. The listed license is Apache-2.0 [7]. The caveat is that compressed checkpoints require separate benchmarking on target hardware; the specified BF16 checkpoint alone needs 51.6 GB of weights [7].

How much BF16 model storage do these open-weight vision-language models need?

BF16 model storage for the listed open-weight vision-language models ranges from 9.3 GB to 360.0 GB, depending on parameter count and precision [12][15].

Qwen3-VL-8B-Instruct (Qwen) needs 17.5 GB of BF16 weights [2]. Qwen2.5-VL-7B-Instruct (Qwen) needs 16.6 GB [9], and Qwen3.5-4B (Qwen) needs 9.3 GB [12].

Parameter counts for those three models are 8.8B [2], 8.3B [9], and 4.7B [12]; context lengths are 256K tokens [2], 128K tokens [9], and 256K tokens [12].

Additional cards list other BF16 footprints: Qwen3.5-9B (Qwen) needs 19.3 GB of BF16 weights [8], gemma-4-26B-A4B-it (google) needs 51.6 GB [7], and google/gemma-4-31B-it (google) needs 62.5 GB [5].

Qwen/Qwen3.8-27B (Qwen) needs 55.6 GB of BF16 weights [1], while Qwen/Qwen3.8-Flash-Next (Qwen) needs 360.0 GB [15]. Use these BF16 weight sizes as the model-storage baseline for local deployment.

Qwen/Qwen3.8-27B (Qwen) has 27.8B parameters and a 256K-token context length [1]. Qwen/Qwen3.8-Flash-Next (Qwen) has 180B parameters and a 256K-token context length [15].

The remaining entries list non-BF16 weight sizes: moonshotai/Kimi-K3 (moonshotai) lists U8 weights at 1560.9 GB [4], and zai-org/GLM-5.3-Flash (zai-org) lists F8_E4M3 weights at 328.3 GB [13].

Which licenses apply to these local model candidates?

The applicable licenses are Apache-2.0 [1][2][5][7][8][9][12], MIT [13], kimi-k3 [4], and qwen-community-1.0 [15]. The Hugging Face model cards dated 2026-09-25 list the license field for each local-model candidate [1][2][4][5][7][8][9][12][13][15].

Seven Hugging Face model cards dated 2026-09-25 list Apache-2.0 in the license field [1][2][5][7][8][9][12]. The Qwen model Qwen3-VL-8B-Instruct (Qwen) [2] was first published on Hugging Face on 2025-10-11 [2]. The Qwen model Qwen2.5-VL-7B-Instruct (Qwen) [9] was first published on Hugging Face on 2025-01-26 [9].

The Google model google/gemma-4-31B-it [5] was first published on Hugging Face on 2026-03-11 [5]. The Google model gemma-4-26B-A4B-it (google) [7] was first published on Hugging Face on 2026-03-11 [7]. The Qwen model Qwen/Qwen3.8-27B [1] was first published on Hugging Face on 2026-08-05 [1].

The Qwen model Qwen3.5-9B (Qwen) [8] was first published on Hugging Face on 2026-02-27 [8]. The Qwen model Qwen3.5-4B (Qwen) [12] was first published on Hugging Face on 2026-02-27 [12].

The zai-org model zai-org/GLM-5.3-Flash [13] was first published on Hugging Face on 2026-08-25 [13] and is MIT [13]. The moonshotai model moonshotai/Kimi-K3 [4] was first published on Hugging Face on 2026-06-13 [4] and uses the kimi-k3 license [4].

The Qwen model Qwen/Qwen3.8-Flash-Next [15] was first published on Hugging Face on 2026-08-24 [15] and uses qwen-community-1.0 [15]. Each Hugging Face model card lists that label in its license field [1][2][4][5][7][8][9][12][13][15].

What context lengths do these open-weight vision-language models support?

The listed open-weight vision-language model candidates support context windows of 128K tokens [9], 256K tokens [1][2][5][7][8][12][15], and 1,024K tokens [4][13]. Qwen/Qwen3.8-27B (Qwen), Qwen3-VL-8B-Instruct (Qwen), google/gemma-4-31B-it (google), gemma-4-26B-A4B-it (google), Qwen3.5-9B (Qwen), Qwen3.5-4B (Qwen), and Qwen/Qwen3.8-Flash-Next (Qwen) each support 256K-token context windows [1][2][5][7][8][12][15]. moonshotai/Kimi-K3 (moonshotai) and zai-org/GLM-5.3-Flash (zai-org) support 1,024K-token context windows [4][13]. Qwen2.5-VL-7B-Instruct (Qwen) supports a 128K-token context window [9]. The 256K-token group includes Apache-2.0 licensed Qwen/Qwen3.8-27B (Qwen) [1], Qwen3-VL-8B-Instruct (Qwen) [2], google/gemma-4-31B-it (google) [5], gemma-4-26B-A4B-it (google) [7], Qwen3.5-9B (Qwen) [8], and Qwen3.5-4B (Qwen) [12]; Qwen/Qwen3.8-Flash-Next (Qwen) uses qwen-community-1.0 [15]. moonshotai/Kimi-K3 (moonshotai) lists the kimi-k3 license [4], and zai-org/GLM-5.3-Flash (zai-org) lists MIT [13]. Qwen3.5-4B (Qwen) lists BF16 weights at 9.3 GB [12], google/gemma-4-31B-it (google) lists 62.5 GB [5], and Qwen3-VL-8B-Instruct (Qwen) lists 17.5 GB [2]. First publication dates include Qwen3-VL-8B-Instruct (Qwen), first published on Hugging Face on 2025-10-11 [2], and Qwen/Qwen3.8-Flash-Next (Qwen), first published on 2026-08-24 [15]. Download counts for the last 30 days as of 2026-09-25 [2][7] include Qwen3-VL-8B-Instruct (Qwen) at 19,149,628 [2] and gemma-4-26B-A4B-it (google) at 11,073,424 [7].

Frequently Asked Questions

Which compact vision-language model has published weight-size detail useful for local sizing?

Qwen's Qwen3-VL-8B-Instruct (Qwen) has 8.8 billion parameters and BF16 weights of 17.5 GB [2]. The same model supports a 256K token context window [2]. Its Apache-2.0 license appears in the model card [2]. The model card lists weight sizes, not quantized-size figures, so local sizing should start from those published values.

What benchmark scores are available for a locally relevant Qwen vision model?

Qwen's Qwen2.5-VL-7B-Instruct (Qwen) has OpenCompass OpenVLM scores as of 2025-09-17: MMMU_VAL 58.0, MMBench_TEST_EN 83.2, and MathVista 68.1 [11]. The model card lists 8.3B parameters, a 128K token context window, BF16 weights of 16.6 GB, and an Apache-2.0 license [9]. The listed entry shows those scores, not DocVQA or ChartQA [11].

Which candidate model cards list a named open-weight license?

Qwen's Qwen3-VL-8B-Instruct (Qwen) and Qwen's Qwen2.5-VL-7B-Instruct (Qwen) list Apache-2.0 [2][9]. zai-org's zai-org/GLM-5.3-Flash lists MIT [13]. google's google/gemma-4-31B-it and google's gemma-4-26B-A4B-it (google) also list Apache-2.0 [5][7]. Card weight sizes include 17.5 GB BF16 for the first model [2] and 328.3 GB F8_E4M3 for GLM-5.3-Flash [13]. The first model supports a 256K token window [2].

What context-window details are available for local planning?

Qwen's Qwen3-VL-8B-Instruct (Qwen) lists a 256K token context window [2]. Qwen's Qwen2.5-VL-7B-Instruct (Qwen) lists 128K tokens [9]. zai-org's zai-org/GLM-5.3-Flash and moonshotai's moonshotai/Kimi-K3 list 1024K tokens [13][4]. google's google/gemma-4-31B-it and Qwen's Qwen/Qwen3.8-27B also list 256K tokens [5][1]. Context length and card-listed weight-size figures are separate fields [1][2][4][5][9][13].

Which candidate models have high card-listed weight sizes?

zai-org's zai-org/GLM-5.3-Flash has F8_E4M3 weights of 328.3 GB [13]. moonshotai's moonshotai/Kimi-K3 has U8 weights of 1,560.9 GB [4]. Qwen's Qwen/Qwen3.8-Flash-Next has BF16 weights of 360.0 GB [15]. google's google/gemma-4-31B-it and Qwen's Qwen/Qwen3.8-27B have BF16 weights of 62.5 GB and 55.6 GB, respectively [5][1]. The cards list these weight-size figures as of 2026-09-25 [1][4][5][13][15].

What download activity can be checked for recent candidate models?

Qwen's Qwen3-VL-8B-Instruct (Qwen) shows 19,149,628 downloads over the last 30 days, as of 2026-09-25 [2]. google's gemma-4-26B-A4B-it (google) shows 11,073,424 [7], and Qwen's Qwen3.5-9B (Qwen) shows 9,781,405 [8]. Qwen's Qwen2.5-VL-7B-Instruct (Qwen) shows 6,530,663 [9], while Qwen's Qwen3.5-4B (Qwen) shows 7,029,433 [12]. Card figures support local popularity checks in this set.

Sources

  1. Qwen/Qwen3.8-27B model card (Hugging Face) — 2026-09-25
  2. Qwen/Qwen3-VL-8B-Instruct model card (Hugging Face) — 2026-09-25
  3. Qwen3 Technical Report — 2025-05-14
  4. moonshotai/Kimi-K3 model card (Hugging Face) — 2026-09-25
  5. google/gemma-4-31B-it model card (Hugging Face) — 2026-09-25
  6. Gemma 4 Technical Report — 2026-07-02
  7. google/gemma-4-26B-A4B-it model card (Hugging Face) — 2026-09-25
  8. Qwen/Qwen3.5-9B model card (Hugging Face) — 2026-09-25
  9. Qwen/Qwen2.5-VL-7B-Instruct model card (Hugging Face) — 2026-09-25
  10. YaRN: Efficient Context Window Extension of Large Language Models — 2023-08-31
  11. OpenCompass OpenVLM Leaderboard (data 2025-09-17) — 2025-09-17
  12. Qwen/Qwen3.5-4B model card (Hugging Face) — 2026-09-25
  13. zai-org/GLM-5.3-Flash model card (Hugging Face) — 2026-09-25
  14. GLM-5: from Vibe Coding to Agentic Engineering — 2026-02-17
  15. Qwen/Qwen3.8-Flash-Next model card (Hugging Face) — 2026-09-25

Related on Q4KM

Get these models on a hard drive

Skip the downloads. Browse our catalog of 985+ commercially-licensed AI models, available pre-loaded on high-speed drives.

Browse Model Catalog

More from the blog

Best Quantization Formats in 2026: GGUF, AWQ, GPTQ, EXL2
2026-09-25 · 16 min read
Best Local AI Runtimes in 2026: llama.cpp, vLLM, Ollama
2026-09-25 · 18 min read
Best Open-Source LLMs to Run Locally in 2026
2026-09-25 · None min read
Best AI Models of July 2026: Complete Rankings by Benchmark, Use Case, and Price
2026-07-09 · 6 min read