Maxwell-V1.0-Flash

Maxwell

MoE decoder-only transformer, dense GQA with Maxwell Block Sparse Attention (MBSA), 3 dense FFN + 42 MoE layers

Open Weights Commercial Use Multimodal F32, BF16 Maxwell multi th lo my km shn

Parameters

37.0B / 3.0B active

MoE total / active

Context

262,144 tokens

native

Modalities

text text

input → output

Benchmarks

#12/137

better than 91% of all scored models

Size class

#2/20

better than 90% of similar-size models

License

Apache License 2.0

About

Maxwell-V1.0-Flash (repo maxwell-model/Maxwell-V1-Flash) is a next-generation, high-performance Mixture-of-Experts (MoE) language model from Maxwell, a new lab publishing under the maxwell-model organization, designed for strong reasoning, agentic and long-context capability at a practical inference cost. It pairs 35B total and 3B active parameters per token (both excluding embeddings; the safetensors shards carry 37B including the 200,704-token vocabulary) with support for context lengths up to 256K tokens (262,144 configured; config.json sets max_position_embeddings to 524,288). Weights were verified on the Hugging Face main branch on November 13, 2026, first community discussion appears November 16, and tech-press coverage around November 18, 2026. The model is text-only, released under Apache License 2.0 with thinking enabled by default in the bundled chat template.

Key features

  • Maxwell Block Sparse Attention (MBSA). Each KV group keeps fixed coverage of 8 prefix and 8 local 128-token blocks and adds up to 16 remote blocks chosen by a learned indexer, bounding main attention at 4,096 tokens per KV group while the full-history KV cache stays available. The MBSA kernel is 6.7x faster than dense GQA at 256K.
  • Manifold-Constrained Hyper-Connections. Multiple residual streams with learned, input-dependent aggregation and constrained mixing, widening information flow without extra attention or feed-forward evaluations. The spec table lists 4 residual streams, 20 Sinkhorn-Knopp iterations and an initial input-mapping scale of 0.01.
  • Fine-Grained Mixture of Experts. A mostly-MoE backbone pairing fine-grained routed experts with always-active shared experts, using a dynamic routing correction bias alongside a lightweight sequence-wise load-balancing loss.

Architecture

45 Transformer layers: 3 dense FFN layers (intermediate size 6,144) followed by 42 MoE layers with 256 routed experts, 12 active per token and 2 shared experts per layer (intermediate size per expert 512; the router uses sigmoid score-function routing with expert-bias correction). Hidden size 2,048. Attention is dense GQA with 32 query heads and 4 KV heads (head dimension 128) with a 16K-token dense-GQA query-position prefix; beyond the prefix, MBSA block-sparse attention applies in every layer, with a 128-token block size and a maximum block budget of 32 per KV group (8 prefix / 8 local / 16 remote). RoPE base is 20,000,000 in config.json (the spec table prints 10,000,000). One auxiliary Multi-Token Prediction (MTP) layer sits outside the backbone. qk_layernorm is enabled.

Pre-training and tokenizer

Pre-training used approximately 26T tokens of multilingual data with the Muon optimizer on most hidden weight matrices. Training starts dense; block-sparse attention enters at 128K mid-training, once the indexer is warmed up against dense full-history attention, then trains jointly with the backbone. A script-aware multilingual tokenizer stays compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan. Long-context training extends the sequence length in stages from 4K to 256K.

Post-training

Two-stage all-domain SFT builds general, reasoning, and agentic capabilities, then targets harder reasoning problems and agent trajectories. Independent RL branches cover general tasks, verifiable reasoning (mathematics, science, logic, code), and agentic tasks (search, terminal use, function calling, software engineering). Multi-teacher on-policy distillation then merges them into one model by supervising student-generated responses.

Evaluation

In the card's comparison table against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, Nemotron-3.5-Lightning and Muse-Glimmer-30B (Maxwell's unified internal evaluation framework), Maxwell-V1.0-Flash leads most rows: Terminal-Bench 2.1 (Terminus-2 harness) 58.65, SWE-Bench-Verified (Claude Code harness) 65.40, MCP-Mark 37.30, Tau3-Bench 67.37, BrowseComp 76.30 (82.20 with context management, marked *), HLE w/Tools 46.10, WideSearch 74.40, DeepSearchQA 83.30, AIME 2025 pass@8 100.00, AIME 2026 pass@8 96.67, HMMT February 2026 pass@8 100.00, IMO-AnswerBench pass@8 95.50, LiveCodeBench v6 pass@8 93.74, GPQA-Diamond 86.52, MMLU-Pro 85.46, MMLU-Redux 93.59, BBH 91.02, long-context RULER 95.81, AA-LCR 75.60, LongBench V2 59.40, LongMemEval 86.18, MultiChallenge 72.20 and MGSM 93.0. The card states post-training evaluation settings explicitly: mostly temperature 0.6, top-p 0.95, top-k 20 at a 32K-token context; AA-LCR, mathematical benchmarks and HLE use 128K; scientific benchmarks 80K; LiveCodeBench v6 uses 64K and multilingual evaluation caps generation at 16K tokens; SWE-Bench Verified runs in the Claude Code harness (version 2.1.212) in a network-isolated environment with a 256K-token window and a four-hour wall-clock limit; Terminal-Bench 2.1 runs in the Harbor framework with the Terminus-2 harness.

Deployment

Ships with a vLLM OpenAI-compatible Docker image (maxwellmodel/vllm-openai:v0.30.0) requiring NVIDIA driver >= 580 (CUDA 13.0) and --trust-remote-code --block-size 128, with a qwen3 reasoning parser and qwen3_coder tool-call parser and --enable-auto-tool-choice. OpenAI-compatible tool definitions are accepted through the tools field, and multi-turn conversations pass reasoning_content and tool_calls back in the assistant message. Recommended sampling: temperature 0.6, top-p 0.95, top-k 20. A hosted playground is available at maxwell-llm.ai/playground, with quickstart docs at maxwell-llm.ai/docs/quickstart.

Links

  • Homepage: https://maxwell-llm.ai/ - Playground: https://maxwell-llm.ai/playground
  • GitHub: https://github.com/maxwell-model/Maxwell-V1-Flash - Technical report (PDF): https://github.com/maxwell-model/Maxwell-V1-Flash/blob/main/Maxwell_v1_flash_tech_report.pdf
  • Quickstart docs: https://maxwell-llm.ai/docs/quickstart - Contact: contact@maxwell-llm.ai

Training Data Pre-trained on ~26T multilingual tokens with the Muon optimizer on most hidden weight matrices; training starts dense and Maxwell Block Sparse Attention (MBSA) enters at 128K mid-training; long-context training extends sequence length in stages from 4K to 256K. Post-training: two-stage all-domain SFT, independent RL branches (general, verifiable reasoning, agentic), multi-teacher on-policy distillation.

Benchmark Scores

Benchmark Score Date Standing
AIME 2025
stem_reasoning
100.00%
—
#1/28
better than 93%
HMMT Feb 25
stem_reasoning
100.00%
—
#1/19
better than 100%
HMMT Nov 25
stem_reasoning
100.00%
—
#1/22
better than 100%
HMMT Feb 26
stem_reasoning
100.00%
—
#1/46
better than 100%
MMLU-Redux
knowledge
91.55%
—
#8/35
better than 79%
INCLUDE
multilingual
92.57%
—
#2/18
better than 88%
MT-RAG
general_agent
9.26%
—
#3/4
better than 33%
HLE (text-only)
stem_reasoning
17.91%
—
#4/7
better than 50%
BIG-Bench Hard
reasoning
100.00%
—
#1/12
better than 100%
2WikiMultihopQA-300
general_agent
100.00%
—
#1/4
better than 100%

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.0B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 74.0 GB 0.5 GB 78.2 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 37.0 GB 0.2 GB 39.1 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 18.5 GB 0.1 GB 19.5 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 18.5 GB 0.1 GB 19.5 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 78.2 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

Architecture

Decoder Block input Embedding vocab 201K · d 2048 Full Attention GQA 32:4 ×45 MoE FFN 256 experts · top-12 · dᴻ 512 Final Norm LM Head vocab 201K output

decoder-only MoE transformer

MoE
Parameters
37B
Active params
3B
Layers
45 45× Full Attention
Attention
GQA 32:4 Grouped Query Attention
Experts
256 top-12
Hidden
2048
Vocabulary
201K
Context
524K
silu BF16 RoPE θ 20M sigmoid routing…

Source: Hugging Face config.json · MaxwellMoeForCausalLM · model repo

Training Pipeline

  1. 1
    pretraining

    Pre-training

    The model is pre-trained on approximately 26T tokens of multilingual data, using the Muon optimizer on most hidden weight matrices. Training starts dense, i.e., the early phase runs with dense attention across the backbone. The block-sparse attention mechanism is deliberately held back and only introduced later during mid-training.

  2. 2
    cpt

    Mid-training: block-sparse attention (MBSA)

    During mid-training, block-sparse attention (Maxwell Block Sparse Attention, MBSA) enters at a 128K context position, once the learned indexer has been warmed up against dense full-history attention. From that point on, the block-sparse attention trains jointly with the rest of the backbone instead of the initial dense-attention setup.

  3. 3
    cpt

    Long-context training

    Long-context training then extends the sequence length in stages from 4K up to 256K, yielding the model's configured 262,144-token context length. This phase follows the main pre-training run and the script-aware multilingual tokenizer keeps the vocabulary compact across languages, especially low-resource Southeast Asian ones such as Thai, Lao, Myanmar, Khmer, and Shan.

  4. 4
    sft

    Two-stage all-domain SFT

    Post-training begins with a two-stage all-domain SFT. The first stage builds general, reasoning, and agentic capabilities across domains, and the second stage targets harder reasoning problems and agent trajectories.

  5. 5
    rl

    Independent RL branches

    Independent reinforcement learning branches then cover three task families: general tasks, verifiable reasoning (mathematics, science, logic, and code), and agentic tasks (search, terminal use, function calling, and software engineering). The branches are trained independently and are subsequently merged into a single model in the following stage.

  6. 6
    other

    Multi-teacher on-policy distillation

    Multi-teacher on-policy distillation finally merges the independent RL branches into one model. The distillation supervises student-generated responses, meaning the student model's own outputs serve as the basis for the teachers' supervision during this merging step.

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Maxwell-V1.0-Flash — here is where to go next.