Maxwell-V1.0-Flash
MoE decoder-only transformer, dense GQA with Maxwell Block Sparse Attention (MBSA), 3 dense FFN + 42 MoE layers
Parameters
37.0B / 3.0B active
MoE total / active
Context
262,144 tokens
native
Modalities
text text
input → output
Benchmarks
#12/137
better than 91% of all scored models
Size class
#2/20
better than 90% of similar-size models
License
Apache License 2.0
About
Maxwell-V1.0-Flash (repo maxwell-model/Maxwell-V1-Flash) is a next-generation, high-performance Mixture-of-Experts (MoE) language model from Maxwell, a new lab publishing under the maxwell-model organization, designed for strong reasoning, agentic and long-context capability at a practical inference cost. It pairs 35B total and 3B active parameters per token (both excluding embeddings; the safetensors shards carry 37B including the 200,704-token vocabulary) with support for context lengths up to 256K tokens (262,144 configured; config.json sets max_position_embeddings to 524,288). Weights were verified on the Hugging Face main branch on November 13, 2026, first community discussion appears November 16, and tech-press coverage around November 18, 2026. The model is text-only, released under Apache License 2.0 with thinking enabled by default in the bundled chat template.
Key features
- Maxwell Block Sparse Attention (MBSA). Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K.
- Manifold-Constrained Hyper-Connections. Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations. The spec table lists 4 residual streams, 20 Sinkhorn-Knopp iterations and an initial input-mapping scale of 0.01.
- Fine-Grained Mixture of Experts. A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss.
Architecture
45 Transformer layers: 3 dense FFN layers (intermediate size 6,144) followed by 42 MoE layers with 256 routed experts, 12 active per token and 2 shared experts per layer (intermediate size per expert 512; the router uses sigmoid score-function routing with expert-bias correction). Hidden size 2,048. Attention is dense GQA with 32 query heads and 4 KV heads (head dimension 128) with a 16K-token dense-GQA query-position prefix; beyond the prefix, MBSA block-sparse attention applies in every layer, with a 128-token block size and a maximum block budget of 32 per KV group (8 prefix / 8 local / 16 remote). RoPE base is 20,000,000 in config.json (the spec table prints 10,000,000). One auxiliary Multi-Token Prediction (MTP) layer sits outside the backbone. qk_layernorm is enabled.
Pre-training and tokenizer
Pre-training used approximately 26T tokens of multilingual data with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone. A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training extends the sequence length in stages from 4K to 256K.
Post-training
Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses.
Evaluation
In the card's comparison table against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, Nemotron-3.5-Lightning and Muse-Glimmer-30B (Maxwell's unified internal evaluation framework), Maxwell-V1.0-Flash leads most rows: Terminal-Bench 2.1 (Terminus-2 harness) 58.65, SWE-Bench-Verified (Claude Code harness) 65.40, MCP-Mark 37.30, Tau3-Bench 67.37, BrowseComp 76.30 (82.20 with context management, marked *), HLE w/Tools 46.10, WideSearch 74.40, DeepSearchQA 83.30, AIME 2025 pass@8 100.00, AIME 2026 pass@8 96.67, HMMT February 2026 pass@8 100.00, IMO-AnswerBench pass@8 95.50, LiveCodeBench v6 pass@8 93.74, GPQA-Diamond 86.52, MMLU-Pro 85.46, MMLU-Redux 93.59, BBH 91.02, long-context RULER 95.81, AA-LCR 75.60, LongBench V2 59.40, LongMemEval 86.18, MultiChallenge 72.20 and MGSM 93.0. The card states post-training evaluation settings explicitly: mostly temperature 0.6, top-p 0.95, top-k 20 at a 32K-token context; AA-LCR, mathematical benchmarks and HLE use 128K; scientific benchmarks 80K; LiveCodeBench v6 uses 64K and multilingual evaluation caps generation at 16K tokens; SWE-Bench Verified runs in the Claude Code harness (version 2.1.212) in a network-isolated environment with a 256K-token window and a four-hour wall-clock limit; Terminal-Bench 2.1 runs in the Harbor framework with the Terminus-2 harness.
Deployment
Ships with a vLLM OpenAI-compatible Docker image (maxwellmodel/vllm-openai:v0.30.0) requiring NVIDIA driver >= 580 (CUDA 13.0) and --trust-remote-code --block-size 128, with a qwen3 reasoning parser and qwen3_coder tool-call parser and --enable-auto-tool-choice. OpenAI-compatible tool definitions are accepted through the tools field, and multi-turn conversations pass reasoning_content and tool_calls back in the assistant message. Recommended sampling: temperature 0.6, top-p 0.95, top-k 20. A hosted playground is available at maxwell-llm.ai/playground, with quickstart docs at maxwell-llm.ai/docs/quickstart.
Links
- Homepage: https://maxwell-llm.ai/ - Playground: https://maxwell-llm.ai/playground
- GitHub: https://github.com/maxwell-model/Maxwell-V1-Flash - Technical report (PDF): https://github.com/maxwell-model/Maxwell-V1-Flash/blob/main/Maxwell_v1_flash_tech_report.pdf
- Quickstart docs: https://maxwell-llm.ai/docs/quickstart - Contact: contact@maxwell-llm.ai
Training Data Pre-trained on ~26T multilingual tokens with the Muon optimizer on most hidden weight matrices; training starts dense and Maxwell Block Sparse Attention (MBSA) enters at 128K mid-training; long-context training extends sequence length in stages from 4K to 256K. Post-training: two-stage all-domain SFT, independent RL branches (general, verifiable reasoning, agentic), multi-teacher on-policy distillation.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
AIME 2025
stem_reasoning
|
100.00%
|
— |
#1/28
better than 93%
|
|
HMMT Feb 25
stem_reasoning
|
100.00%
|
— |
#1/19
better than 100%
|
|
HMMT Nov 25
stem_reasoning
|
100.00%
|
— |
#1/22
better than 100%
|
|
HMMT Feb 26
stem_reasoning
|
100.00%
|
— |
#1/46
better than 100%
|
|
MMLU-Redux
knowledge
|
91.55%
|
— |
#8/35
better than 79%
|
|
INCLUDE
multilingual
|
92.57%
|
— |
#2/18
better than 88%
|
|
MT-RAG
general_agent
|
9.26%
|
— |
#3/4
better than 33%
|
|
HLE (text-only)
stem_reasoning
|
17.91%
|
— |
#4/7
better than 50%
|
|
BIG-Bench Hard
reasoning
|
100.00%
|
— |
#1/12
better than 100%
|
|
2WikiMultihopQA-300
general_agent
|
100.00%
|
— |
#1/4
better than 100%
|
|
LongMemEval
long_context
|
44.52%
|
— |
#2/4
better than 67%
|
|
RULER (32K context)
long_context
|
45.73%
|
— |
#2/4
better than 67%
|
|
BigBench Extra Hard
stem_reasoning
|
60.27%
|
— |
#5/9
better than 50%
|
|
GPQA Diamond
stem_reasoning
|
85.29%
|
— |
#47/84
better than 45%
|
|
SWE-bench Verified
coding_agent
|
74.54%
|
— |
#43/61
better than 30%
|
|
Multi-Challenge
instruction_following
|
100.00%
|
— |
#1/20
better than 100%
|
|
PopQA-300
general_agent
|
100.00%
|
— |
#1/4
better than 100%
|
|
RoleBench
role_playing
|
100.00%
|
— |
#1/4
better than 100%
|
|
TAU3-Bench
general_agent
|
97.74%
|
— |
#5/24
better than 83%
|
|
IMOAnswerBench
stem_reasoning
|
100.00%
|
— |
#1/32
better than 100%
|
|
AIME 26 (mean@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
AIME 26
stem_reasoning
|
96.77%
|
— |
#11/46
better than 78%
|
|
LiveCodeBench v6
stem_reasoning
|
89.30%
|
— |
#18/50
better than 65%
|
|
AA-LCR
long_context
|
90.00%
|
— |
#12/48
better than 77%
|
|
WebWalkerQA-300
general_agent
|
100.00%
|
— |
#1/4
better than 100%
|
|
AIME 2025 (mean@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
WideSearch
general_agent
|
76.71%
|
— |
#16/30
better than 48%
|
|
LiveCodeBench v6 (pass@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
MuSiQue-300
general_agent
|
100.00%
|
— |
#1/4
better than 100%
|
|
IFEval
instruction_following
|
93.87%
|
— |
#12/30
better than 59%
|
|
C-Eval
knowledge
|
78.57%
|
— |
#10/19
better than 50%
|
|
MT-Bench-101
instruction_following
|
85.00%
|
— |
#3/4
better than 33%
|
|
DeepSearch QA
general_agent
|
64.29%
|
— |
#10/17
better than 44%
|
|
MMMLU
multilingual
|
100.00%
|
— |
#1/23
better than 100%
|
|
MCPMark
general_agent
|
37.96%
|
— |
#6/17
better than 69%
|
|
HMMT Feb 25 (mean@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
Terminal-Bench 2.1 (Terminus-2)
coding_agent
|
54.88%
|
— |
#38/56
better than 33%
|
|
CoSER
role_playing
|
45.89%
|
— |
#3/4
better than 33%
|
|
MMLU-Pro
knowledge
|
82.13%
|
— |
#17/50
better than 67%
|
|
Belebele (5-language subset)
multilingual
|
92.80
|
— |
#7/7
better than 0%
|
|
IFBench
instruction_following
|
68.46%
|
— |
#34/46
better than 27%
|
|
HLE (with tools)
stem_reasoning
|
57.40%
|
— |
#35/54
better than 36%
|
|
HotpotQA-300
general_agent
|
100.00%
|
— |
#1/4
better than 100%
|
|
IMO-AnswerBench (mean@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
HMMT Feb 26 (mean@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
HMMT Nov 25 (mean@8)
stem_reasoning
|
100.00%
|
— |
#1/4
better than 100%
|
|
MLogiQA
multilingual
|
100.00%
|
— |
#1/4
better than 100%
|
|
MGSM
multilingual
|
99.78%
|
— |
#2/9
better than 88%
|
|
Bamboogle
general_agent
|
100.00%
|
— |
#1/4
better than 100%
|
|
KOR-Bench
reasoning
|
89.21%
|
— |
#3/7
better than 67%
|
|
BrowseComp
general_agent
|
80.71%
|
— |
#19/53
better than 65%
|
|
Global PIQA
multilingual
|
89.33%
|
— |
#3/10
better than 78%
|
|
AdvancedIF
instruction_following
|
71.30
|
— |
#4/4
better than 0%
|
|
LongBench v2
long_context
|
78.05%
|
— |
#7/26
better than 76%
|
What is a good AIME 2025 score? What is a good HMMT Feb 25 score? What is a good HMMT Nov 25 score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 74.0 GB | 0.5 GB | 78.2 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 37.0 GB | 0.2 GB | 39.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 18.5 GB | 0.1 GB | 19.5 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 18.5 GB | 0.1 GB | 19.5 GB | Which GPUs can run it? Calculate on your GPU |
Only datacenter GPUs — FP16 needs 78.2 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.
Consumer GPUs that fit this model with a single card (INT4):
Architecture
decoder-only MoE transformer
- Parameters
- 37B
- Active params
- 3B
- Layers
- 45 45× Full Attention
- Attention
- GQA 32:4 Grouped Query Attention
- Experts
- 256 top-12
- Hidden
- 2048
- Vocabulary
- 201K
- Context
- 524K
Source: Hugging Face config.json · MaxwellMoeForCausalLM · model repo
Training Pipeline
-
1
pretraining
Pre-training
The model is pre-trained on approximately 26T tokens of multilingual data, using the Muon optimizer on most hidden weight matrices. Training starts dense, i.e., the early phase runs with dense attention across the backbone. The block-sparse attention mechanism is deliberately held back and only introduced later during mid-training.
-
2
cpt
Mid-training: block-sparse attention (MBSA)
During mid-training, block-sparse attention (Maxwell Block Sparse Attention, MBSA) enters at a 128K context position, once the learned indexer has been warmed up against dense full-history attention. From that point on, the block-sparse attention trains jointly with the rest of the backbone instead of the initial dense-attention setup.
-
3
cpt
Long-context training
Long-context training then extends the sequence length in stages from 4K up to 256K, yielding the model's configured 262,144-token context length. This phase follows the main pre-training run and the script-aware multilingual tokenizer keeps the vocabulary compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan.
-
4
sft
Two-stage all-domain SFT
Post-training begins with a two-stage all-domain SFT. The first stage builds general, reasoning, and agentic capabilities across domains, and the second stage targets harder reasoning problems and agent trajectories.
-
5
rl
Independent RL branches
Independent reinforcement learning branches then cover three task families: general tasks, verifiable reasoning (mathematics, science, logic, and code), and agentic tasks (search, terminal use, function calling, and software engineering). The branches are trained independently and are subsequently merged into a single model in the following stage.
-
6
other
Multi-teacher on-policy distillation
Multi-teacher on-policy distillation finally merges the independent RL branches into one model. The distillation supervises student-generated responses, meaning the student model's own outputs serve as the basis for the teachers' supervision during this merging step.
Linked Resources
Maxwell Homepage
https://maxwell-llm.ai/
Maxwell Chat Playground
https://maxwell-llm.ai/playground
maxwell-model/Maxwell-V1-Flash
https://github.com/maxwell-model/Maxwell-V1-Flash
Maxwell-V1.0-Flash Technical Report (PDF)
https://github.com/maxwell-model/Maxwell-V1-Flash/blob/main/Maxwell_v1_flash_tech_report.pdf
Maxwell-V1.0-Flash Quickstart
https://maxwell-llm.ai/docs/quickstart
Trend Analysis
Current
1,703
likes
followers
downloads_all_time
No comments yet
Be the first to share your thoughts!