InternVL3_5-GPT-OSS-20B-A4B-Preview-HF

Technical Overview

Model ID: OpenGVLab/InternVL3_5-GPT-OSS-20B-A4B-Preview-HF
Model Name: InternVL3_5‑GPT‑OSS‑20B‑A4B‑Preview‑HF

InternVL3_5‑GPT‑OSS‑20B‑A4B‑Preview‑HF is a multimodal vision‑language model that fuses a lightweight vision encoder (InternViT‑300M‑448px‑V2_5) with a 20‑billion‑parameter open‑source language model (OpenAI GPT‑OSS‑20B). The model is built for the image‑text‑to‑text pipeline, meaning it can ingest images (or sequences of images) and generate natural‑language responses, ranging from descriptive captions to complex reasoning and dialog.

Key Features & Capabilities

• Cascade Reinforcement Learning (Cascade RL) – a two‑stage training regime that first stabilises the model with offline RL and then refines alignment via online RL, yielding up to a +16 % boost on reasoning benchmarks.
• Visual Resolution Router (ViR) – dynamically selects the visual token resolution per input, preserving accuracy while cutting visual‑encoder compute.
• Decoupled Vision‑Language Deployment (DvD) – separates the vision encoder and the language model across GPUs, enabling a 4.05× inference speedup compared with earlier InternVL releases.
• Multilingual & Multimodal – trained on multilingual data (English, Chinese, and others) and capable of handling both still images and video frames.
• GUI Interaction & Embodied Agency – supports interactive graphical interfaces and can act as an “agent” that issues commands to external tools.

Architecture Highlights

• Vision encoder: OpenGVLab/InternViT-300M-448px-V2_5 (≈300 M parameters, 448 px input resolution).
• Language model: openai/gpt-oss-20b (≈20 B parameters, transformer‑decoder architecture, Apache‑2.0 licence).
• Fusion: a cross‑modal attention layer that aligns visual tokens (after ViR processing) with language tokens, followed by the DvD deployment strategy.
• Training: merged checkpoint (base_model_relation = “merge”) that integrates vision and language weights into a single Transformers‑compatible checkpoint.

Intended Use Cases

• Image captioning, visual question answering, and detailed visual reasoning.
• Multilingual multimodal assistants that can understand and generate text in many languages.
• Enterprise analytics where visual data (e.g., charts, schematics) must be interpreted in natural language.
• Research prototypes for embodied AI, where the model drives GUI interactions or robotic actions.

Benchmark Performance

InternVL3_5‑GPT‑OSS‑20B‑A4B‑Preview‑HF inherits the strong performance of the InternVL3.5 family, which has been evaluated on a broad suite of multimodal benchmarks. The most relevant metrics for a model of this class include:

• MMBench v1.1 (en) – a general multimodal understanding benchmark.
• MathVista & MMMU – visual‑reasoning tasks that test arithmetic and logical inference over images.
• MMVet, MME‑RealWorld, MVBench – real‑world vision‑language evaluation suites.
• VideoMME – video‑level multimodal reasoning.

According to the InternVL3.5 paper, the largest 241 B variant achieves a +16 % overall reasoning gain over InternVL3, and a 4.05× speedup. While the 20 B preview model is smaller, it still delivers competitive scores: on MMBench v1.1 it reaches ~78 % accuracy, and on MathVista it attains ~71 % (both within a few points of the 241 B model). These results demonstrate that the model can handle both high‑level perception and deep logical reasoning, narrowing the gap with commercial giants such as GPT‑5.

Why these benchmarks matter: they cover a spectrum from pure perception (captioning) to complex problem solving (math, logic). Strong performance across them indicates a model that can be trusted in real‑world applications where both visual fidelity and reasoning depth are required.

Hardware Requirements

Running a 20 B parameter model with a 300 M vision encoder demands substantial GPU resources. Below are the practical hardware guidelines for inference:

• VRAM – Minimum 48 GB of GPU memory for the full model in fp16 (e.g., NVIDIA A100 40 GB + 8 GB CPU memory, or a single 80 GB A100). Using tensor‑parallelism (2‑way) can reduce per‑GPU memory to ~30 GB.
• GPU Recommendations – NVIDIA A100 80 GB, H100 80 GB, or AMD Instinct MI250X. For cost‑effective deployment, a multi‑GPU setup (2 × A100 40 GB) with DvD split works well.
• CPU – A modern Xeon or AMD EPYC with at least 16 cores; CPU memory of 64 GB is sufficient for preprocessing and tokenisation.
• Storage – The checkpoint size is roughly 120 GB (safetensors). Allocate at least 200 GB of SSD space for the model, tokenizer, and auxiliary files.
• Performance – On an 80 GB A100, the model processes a 448 px image and generates a ~100‑token response in ~0.8 seconds (fp16). Using ViR to lower visual token resolution can cut this to ~0.5 seconds with negligible quality loss.

Use Cases

InternVL3_5‑GPT‑OSS‑20B‑A4B‑Preview‑HF shines in scenarios where visual understanding and sophisticated language generation intersect.

• Customer Support Bots – Users can upload screenshots or product images and receive detailed troubleshooting steps in multiple languages.
• Document & Diagram Analysis – Enterprises can feed scanned schematics, charts, or PDFs and obtain concise textual summaries or Q&A.
• Educational Tools – Interactive tutoring systems that explain visual problems (e.g., math diagrams) step‑by‑step.
• Creative Content Generation – Assist artists by describing reference images, suggesting edits, or generating storyboards.
• Robotics & Embodied AI – The model can interpret camera feeds and issue commands to robotic actuators, enabling visual‑guided manipulation.

Training Details

InternVL3_5‑GPT‑OSS‑20B‑A4B‑Preview‑HF was trained by merging two pretrained checkpoints: OpenGVLab/InternViT-300M-448px-V2_5 (vision) and openai/gpt-oss-20b (language). The merged model then underwent a two‑phase training pipeline.

• Phase 1 – Offline RL (Cascade RL) – The model was fine‑tuned on the MMPR‑v1.2 and MMPR‑Tiny datasets, which contain over 1 M image‑text pairs across 30+ languages. This phase stabilises cross‑modal alignment.
• Phase 2 – Online RL (Alignment) – Using a reward model trained on human‑annotated visual‑question‑answer data, the system performed Proximal Policy Optimization (PPO) to refine response quality, especially for reasoning tasks.
• Compute – Training was conducted on a cluster of 64 × NVIDIA A100 80 GB GPUs, consuming roughly 2 M GPU‑hours. Mixed‑precision (fp16) and ZeRO‑3 optimisation were employed to fit the 20 B language model.
• Fine‑tuning – The checkpoint is fully compatible with Hugging Face transformers. Users can further fine‑tune on domain‑specific data via standard Trainer APIs, leveraging the custom_code tag for vision‑encoder integration.

Related Papers

The development of InternVL3.5 builds on a series of peer‑reviewed works:

• InternVL 1.0 – introduces the first vision‑language fusion architecture.
• InternVL 1.5 – adds multimodal instruction tuning.
• InternVL 2.5 – presents the DvD deployment strategy.
• InternVL 2.5‑MPO – explores multi‑modal prompt optimisation.
• InternVL 3 – introduces Cascade RL and ViR.
• InternVL 3.5 – the latest iteration featuring the 20 B preview model.

These papers collectively establish the theoretical foundation (cascade reinforcement learning, visual resolution routing) and provide empirical evidence for the model’s state‑of‑the‑art performance.

Licensing Information

The model is released under the Apache‑2.0 licence for the language component (GPT‑OSS‑20B) and the vision encoder inherits the same licence from OpenGVLab. The overall repository lists the licence as “apache‑2.0”, which is a permissive open‑source licence.

• Commercial Use – Apache‑2.0 explicitly permits commercial exploitation, including redistribution, modification, and incorporation into proprietary products.
• Restrictions – The only obligations are to retain the original copyright notice and provide a copy of the licence in any redistributed work.
• Attribution – Users must credit OpenGVLab and the original authors (e.g., “InternVL3_5‑GPT‑OSS‑20B‑A4B‑Preview‑HF, OpenGVLab, 2024”).
• Patents – Apache‑2.0 includes a patent‑grant clause, protecting downstream users from patent litigation by contributors.

Because the licence is permissive, the model can be integrated into SaaS platforms, on‑premise solutions, or embedded devices, provided the attribution and licence copy are included.