Sage-S2.1-Flash
Glm5Next hybrid sparse-MoE: 34 gated linear-attention + 11 DSA full-attention layers, 288 experts (top-8 + 1 shared), native 1M context
Parameters
321.3B
Context
1,048,576 tokens
native
Modalities
text, image, video text
input → output
Benchmarks
#31/134
better than 77% of all scored models
License
MIT License
About
Sage-S2.1-Flash (card title: Sage S2.1 Flash) is a scientific multimodal model from Shanghai Artificial Intelligence Laboratory (the team behind the InternLM model family). It is designed for research tasks that require multiple rounds of analysis and verification, combining scientific data understanding with information retrieval, programming, and tool use. The model can operate specialized software to perform analyses, check results against intermediate outputs and visual feedback, and adjust its subsequent steps accordingly.
Scientific multimodality via SciPerception
Through the unified representation architecture of the companion Sage SciPerception system, the model supports more than ten types of scientific input, including molecular graphs, crystal structures (CIF), and protein sequences and 3D protein structures. Features of each input type are encoded and projected into a shared representation space, where they can be reasoned over alongside natural-language instructions. Supported task styles include prediction, classification, site localization, quantitative analysis, and structure inference.
Scientific encoders with trainable projectors (for example the projectors/sage-s2.1-flash/cif/jarvis crystal projector) map scientific data into the language model's input embedding space; the projector output is combined with the text token embeddings to form the prompt sequence. On the serving side, vLLM runs with --enable-prompt-embeds and accepts prompt_embeds on /v1/completions, using the tokenizer and text embeddings that match the base model.
Backbone architecture (config.json)
config.json declares Glm5NextForConditionalGeneration with model type glm5_next: a hybrid sparse-Mixture-of-Experts (MoE) decoder with 45 layers that mixes gated linear attention and DeepSeek Sparse Attention (DSA) full attention: 34 gated linear-attention layers (the kda_layers / linear_attn_config group - a Kimi Delta Attention-style gated linear attention with short-conv kernel size 4, gate lower bound -5, 64 heads x 128 head dim) alternate with 11 DSA full-attention layers in a repeating 3:1 pattern plus one final linear layer. The DSA layers use MLA-style latent attention components (q_lora_rank 1536, kv_lora_rank 512, qk_nope_head_dim 256, v_head_dim 256, nope-only MLA with qk_rope_head_dim 0) plus a sparse indexer (index_head_dim 128, index_topk 2048, interleaved indexer RoPE) running on every full-attention layer.
MoE routing: 288 routed experts with top-8 per-token selection and 1 shared expert, sigmoid scoring via the noaux_tc top-k method (topk_method: noaux_tc, routed_scaling_factor 2.5, router in float32, norm_topk_prob on), MoE intermediate size 2048 (dense intermediate 12288) and the first 3 layers dense (first_k_dense_replace 3). Additional structural specifics: hyper-connections (mhc: true, hc_mult 4, 20 Sinkhorn iterations) as multi-stream residual connections, one multi-token prediction (MTP) layer (num_nextn_predict_layers 1, indexer shared for MTP iteration), hidden size 4,096, vocab 154,880, tie_word_embeddings false.
The vision tower (glm5_next_vision) is a 24-layer ViT-style encoder (image size 448, patch 14, spatial merge 2, SwiGLU, projection intermediate 10,240) feeding into the 4,096-dim language-model space; both image and video token ids exist in config (image_token_id 154854, video_token_id 154855), so image and video ingestion run through the shared vision stack.
Parameters and native context
Safetensors total: 321,323,031,390 parameters ("321B params"; 321,319,393,536 BF16 + 3,637,854 F32 - the F32 slice matches the float32 router dtype); tensor types F32 and BF16. max_position_embeddings 1,048,576 gives a native 1M context window. A quantized FP8 release ships as internlm/Sage-S2.1-Flash-FP8; the model tree lists 9 community quantizations, and ModelScope hosts mirrors for both releases. Downloads last month: 337.
Evaluation
The card's evaluation covers a 15-benchmark comparison chart plus a 33-row benchmark table against five comparison models: GLM-5.3-Flash, DeepSeek-V4.1-Flash-1017, Qwen3.8-Max, GPT-5.6 Sol, and Claude Opus 5. Headline results: Sage S2.1 Flash placed first on MLE-bench Lite (90.3, ahead of DeepSeek-V4.1-Flash-1017 at 89.2**)**, OfficeQA Pro (72.2), Harvey LAB (25.0) and SpreadsheetBench 2 (37.1), and second on AutomationBench, ALE-CLI, BFCL V4, WebArena-Verified, DeepResearch Bench II, JobBench, and OSWorld-Verified. Agentic coding reaches 87.9 on Terminal-Bench 2.1 and 67.9 on DeepSWE; search and deep-research runs hit 91.2 on BrowseComp, 94.4 on DeepSearchQA, 81.5 on WideSearch, and 54.2 on DeepResearch Bench II; computer-use and web tasks reach 85.2 on OSWorld-Verified, 67.6 on WebArena-Verified, 78.0 on Vision2Web, 84.8 on OSWorld-G and 75.8 on BabyVision.
Deployment
Official deployment guides cover the BF16 and FP8 releases for general tasks (coding, office productivity, security, computer use); scientific workloads use the projector flow above with shared loading scripts and the inference guide from the companion internlm/Sage-SciPerception repository (load_projector.py, modeling_projector.py, vLLM --enable-prompt-embeds). The model is not served by any HF Inference Provider.
License and release
Released on HuggingFace on 2026-11-15. The code and model weights are released under the MIT License - open weights, fully permissive, commercial use allowed.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
Workspace-Bench-Lite
general_agent
|
90.65%
|
— |
#3/6
better than 60%
|
|
BFCL-V4
general_agent
|
89.88%
|
— |
#2/33
better than 97%
|
|
MLS-Bench
coding_agent
|
21.21%
|
— |
#4/6
better than 40%
|
|
PaperBench
coding_agent
|
88.99%
|
— |
#6/15
better than 64%
|
|
Vision2Web
vision_language
|
94.23%
|
— |
#2/9
better than 88%
|
|
BrowseComp
general_agent
|
97.13%
|
— |
#4/50
better than 92%
|
|
PostTrainBench
coding_agent
|
89.08%
|
— |
#6/24
better than 78%
|
|
GDPVal
general_agent
|
86.45%
|
— |
#6/26
better than 80%
|
|
WorkSpaceBench
general_agent
|
20.83%
|
— |
#11/15
better than 29%
|
|
WebArena-Verified
general_agent
|
96.46%
|
— |
#2/11
better than 90%
|
|
OSWorld-Verified
general_agent
|
97.64%
|
— |
#2/31
better than 97%
|
|
OSWorld V2.1
general_agent
|
21.97%
|
— |
#4/7
better than 33%
|
|
Agents' Last Exam (CLI)
general_agent
|
39.18%
|
— |
#3/8
better than 71%
|
|
Automation-Bench
general_agent
|
88.27%
|
— |
#3/29
better than 93%
|
|
DeepresearchBII
general_agent
|
43.10%
|
— |
#3/7
better than 67%
|
|
SWE-bench Pro
coding_agent
|
67.56%
|
— |
#18/81
better than 78%
|
|
OfficeQA Pro
general_agent
|
100.00%
|
— |
#1/17
better than 100%
|
|
WideSearch
general_agent
|
90.61%
|
— |
#10/28
better than 67%
|
|
WeaveBench
vision_language
|
29.41%
|
— |
#4/5
better than 25%
|
|
BabyVision
vision_language
|
77.55%
|
— |
#9/27
better than 69%
|
|
MLE-bench Lite
coding_agent
|
100.00%
|
— |
#1/6
better than 100%
|
|
TauBench V3 Banking
general_agent
|
79.22%
|
— |
#5/23
better than 82%
|
|
DeepSearch QA
general_agent
|
97.32%
|
— |
#3/14
better than 85%
|
|
SkillsBench
general_agent
|
72.75%
|
— |
#7/14
better than 54%
|
|
SpreadsheetBench 2
general_agent
|
100.00%
|
— |
#1/6
better than 100%
|
|
JobBench
general_agent
|
80.09%
|
— |
#4/23
better than 86%
|
|
Apex-Agents
general_agent
|
87.38%
|
— |
#4/27
better than 88%
|
|
Terminal-Bench 2.1 (Terminus-2)
coding_agent
|
97.52%
|
— |
#5/54
better than 91%
|
|
NL2Repo-Bench
coding_agent
|
66.29%
|
— |
#9/26
better than 68%
|
|
Harvey Legal Agent
general_agent
|
100.00%
|
— |
#1/7
better than 100%
|
|
OSWorld-G
general_agent
|
86.03%
|
— |
#2/6
better than 80%
|
|
DeepSWE
coding_agent
|
91.88%
|
— |
#6/26
better than 80%
|
|
BrowseComp-VL
vision_language
|
60.50
|
— |
#3/3
better than 0%
|
What is a good Workspace-Bench-Lite score? What is a good BFCL-V4 score? What is a good MLS-Bench score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 642.6 GB | 49.9 GB | 724.6 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 321.3 GB | 24.9 GB | 362.3 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 160.7 GB | 12.5 GB | 181.2 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 160.7 GB | 12.5 GB | 181.2 GB | Which GPUs can run it? Calculate on your GPU |
Only datacenter GPUs — FP16 needs 724.6 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.
Architecture
hybrid linear-attention + sparse (DSA) full-attention MoE transformer decoder
- Parameters
- 321.3B
- Layers
- 45 34× Linear / Recurrent + 11× Full Attention
- Attention
- MLA Sparse Attention
- Experts
- 288 top-8 + 1 shared
- Hidden
- 4096
- Vocabulary
- 155K
- Context
- 1M
Source: Hugging Face config.json · Glm5NextForConditionalGeneration · exact layer pattern · model repo
Linked Resources
Sage S2.1 Flash Official Website
https://internlm.github.io/Sage-S2.1-Flash/
Sage-S2.1 (HF Collection)
https://huggingface.co/collections/internlm/sage-s21
internlm/Sage-SciPerception (companion scientific-perception model repo)
https://huggingface.co/internlm/Sage-SciPerception
internlm/Sage-S2.1-Flash-FP8 (companion FP8 quantization repo)
https://huggingface.co/internlm/Sage-S2.1-Flash-FP8
Sage-S2.1-Flash on ModelScope
https://www.modelscope.cn/models/Shanghai_AI_Laboratory/Sage-S2.1-Flash
Sage-S2.1-Flash-FP8 on ModelScope
https://www.modelscope.cn/models/Shanghai_AI_Laboratory/Sage-S2.1-Flash-FP8
Sage S2.1 Flash Deployment Guide
https://huggingface.co/internlm/Sage-S2.1-Flash/blob/main/deployment_guide.md
Sage S2.1 Flash FP8 Deployment Guide
https://huggingface.co/internlm/Sage-S2.1-Flash-FP8/blob/main/deployment_guide.md
Ask for provider support (InferenceSupport discussion #13597)
https://huggingface.co/spaces/huggingface/InferenceSupport/discussions/13597
No comments yet
Be the first to share your thoughts!