Edge0-120B-NVFP4

Edge0-120B · NVFP4 (Edge0/Edge0-120B-NVFP4) is a 120B-parameter latent sparse MoE agent model for local single-GPU inference, published by Edge0 (GitHub org Edge0-AI) on 12 November 2026 together with Edge0-145B in the Edge0 v0.2.0 drop. The HF model tree lists Qwen/Qwen3.8-Flash-Next as base model; Edge0-specific contributions ship under the Edge0 Community Model License 1.0, and the release extends the open-source Strata engine with latent-MoE support. Downloads stood at 3,806 in the month of ingestion.

Architecture

A latent sparse MoE stack: 48 layers of hybrid attention — full attention every 4th layer plus a sparse indexer — with 512 experts per layer, of which 10 routed + 1 shared are active per token (SwiGLU expert width 640). Total parameters are 120B with 8.64B active per token. The defining trick is the latent expert width: only the 512 experts work at latent width ℓ = 1280 (expert pool 60.4B parameters), while everything else — attention, token embedding, lm_head, and the shared D/U projections — stays at full width 2560. Each expert is a fixed 1,164,800 B record, so the engine keeps a few thousand experts on the GPU and streams the remainder from host RAM or SSD ("expert residency"), making cost track resident experts rather than model size. A PLE component adds a 51.2B fp8 n-gram embedding table that is memory-mapped, and a 1.26B MTP draft head (512 experts) enables speculative decode with a configurable verify window.

Context is 256K with KV stored in int8 (32K cells resident in VRAM, K/V streamed from pinned host RAM). The repository is a native-4-bit (NVFP4) build at 32.96 GiB; a sibling Q3_K build ships at 27.98 GiB (the GGUF sidebar label "5B params" is a metadata artifact — the card states 120B total).

Performance and speed

On RTX 5090 D 24 GB with NVFP4 the card reports 4,357.1 tok/s 16K prefill, 2,668.5 tok/s 4K prefill, 124.9 tok/s chat decode and 135.5 tok/s agent decode; the Q3_K build reaches 144.6 tok/s agent decode. A constrained budget test (16 GiB VRAM + 20 GiB host RAM, cgroup-capped) still delivers 59.9–73.3 tok/s agent decode, and Apple M2 Ultra (Metal, Q3_K) 63.0 tok/s.

Quality headline: 75.3 Terminal-Bench 2.1. The six-benchmark quality chart (DeepSWE 1.1, Terminal-Bench 2.1, HLE, GPQA-Diamond, IFBench, AA-LCR) against Edge0-145B, Qwen3.8-Flash-Next, Qwen3.8-27B, DeepSeek-V4-Flash-0731 and Claude-Opus-4.6 (Max) shows Edge0-120B at 51.3 DeepSWE 1.1, 31.0 HLE, 87.4 GPQA-Diamond, 79.3 IFBench and 81.0 AA-LCR.

Stack and deployment

The Edge0 stack comprises edge0-strata (C++/GGML inference engine), edge0-infer (CLI run/chat/bench/serve/info plus OpenAI- and Anthropic-compatible local endpoints) and the Edge0 Chat desktop app (Windows, macOS). The NVFP4 build is CUDA-only; Q3_K also runs on Metal. Model packs are identified by model.json and expose VRAM/RAM capacity controls (--vram-reserve-mib, --expert-cache, --kv-resident, --resident-budget-gib, --mmap-experts for SSD offload).

License

Edge0-specific contributions: Edge0 Community Model License 1.0 (personal use, research, modification, noncommercial sharing, internal use, and commercial use of generated outputs; selling/commercial bundling, commercial model services, or independent commercial coding/office assistants require a separate written license). Qwen materials remain under the Qwen Community License 1.0; framework code Apache-2.0; Strata MIT. The engine repo description tags the family "device-native foundation models".