Edge0-145B-NVFP4

Edge0-145B · NVFP4 (Edge0/Edge0-145B-NVFP4) is a 145B-parameter latent sparse MoE agent model for local single-GPU inference, published by Edge0 (GitHub org Edge0-AI) on 12 November 2026 together with Edge0-120B in the Edge0 v0.2.0 drop. The HF model tree lists Qwen/Qwen3.8-Flash-Next as base model; Edge0-specific contributions ship under the Edge0 Community Model License 1.0, and the release extends the open-source Strata engine with latent-MoE support. Downloads stood at 33,819 in the month of ingestion.

Architecture

A latent sparse MoE stack: 48 layers of hybrid attention — full attention every 4th layer plus a sparse indexer — with 512 experts per layer, of which 10 routed + 1 shared are active per token (SwiGLU expert width 640). Total parameters are 145B with 10.12B active per token. The defining trick is the latent expert width: only the 512 experts work at latent width ℓ = 1792 (expert pool 84.6B parameters), while everything else — attention, token embedding, lm_head, and the shared D/U projections — stays at full width 2560. D projects the token down to ℓ per routing group, the router picks 10 of 512 experts, each expert runs ℓ → 640 → ℓ, and U projects the combined result back to 2560. Each expert is a fixed 1,630,720 B record, so the engine keeps a few thousand experts on the GPU and streams the remainder from host RAM or SSD (expert residency), making cost track resident experts rather than model size. A PLE component adds a 51.2B fp8 n-gram embedding table that is memory-mapped, and a 1.26B MTP draft head (512 experts) enables speculative decode with a configurable verify window (shipped packs use --spec 4).

Context is 256K with KV stored in int8 (32K cells resident in VRAM, K/V streamed from pinned host RAM, 1.55 GiB at 32768 cells). The repository is a native-4-bit (NVFP4) build at 47.79 GiB; a sibling Q3_K build ships at 40.81 GiB (the GGUF sidebar label "5B params" is a metadata artifact — the card states 145B total).

Performance and speed

On RTX 5090 D 24 GB with NVFP4 the card reports 3,165.7 tok/s 16K prefill, 1,839.6 tok/s 4K prefill, 79.6 tok/s chat decode and 98.3 tok/s agent decode; the Q3_K build on the same card reaches 3,554.9 tok/s 16K prefill and 121.2 tok/s agent decode. A constrained budget test (16 GiB VRAM + 20 GiB host RAM, cgroup-capped) still delivers 335 tok/s 16K prefill and 44.9 tok/s agent decode; Apple M2 Ultra (Metal, Q3_K) reaches 57.0 tok/s agent decode. With residency enabled the NVFP4 build held 23.25 GiB VRAM + 38.27 GiB host RAM on the RTX 5090 D 24 GB.

Quality headline (six-benchmark chart against Edge0-120B, Qwen3.8-Flash-Next, Qwen3.8-27B, DeepSeek-V4-Flash-0731 and Claude-Opus-4.6 (Max)): DeepSWE 1.1 64.6 and Terminal-Bench 2.1 86.5 (the agent & coding panels, both row-best) with HLE 35.4, GPQA-Diamond 90.4, IFBench 80.6 and AA-LCR 79.3 — competitive with much larger frontier models while running on one consumer GPU.

Stack and deployment

The Edge0 stack comprises edge0-strata (the inference engine), edge0-infer (CLI serve/chat plus OpenAI- and Anthropic-compatible local endpoints) and the Edge0 Chat desktop app (Windows). The NVFP4 build is CUDA-only (CUDA 13, Linux/Windows x64/ARM64); Q3_K also runs on Metal. Model packs are identified by model.json and expose VRAM/RAM capacity controls: --vram-reserve-mib, --expert-cache (auto derives slots from free VRAM after the fixed footprint: skeleton projections ≈ 2.9 GiB, MTP draft ≈ 0.8 GiB, verify/draft-head buffers ≈ 0.45 GiB), --kv-resident, --resident-budget-gib, --mmap-experts for SSD offload, --ple-io direct|mmap|ram for the PLE table, and --mtp-q4 for 4-bit draft projections. A third-party community build (infosave/Edge0-145B-cmf for the Rust cortiq engine) copies the NVFP4 routed experts bit for bit and re-encodes D/U, shared expert and router — not made or endorsed by Edge0.

License

Edge0-specific contributions: Edge0 Community Model License 1.0 (personal use, research, modification, noncommercial sharing, internal use, and commercial use of generated outputs; selling/commercial bundling, commercial model services, or independent commercial coding/office assistants require a separate written license). Qwen materials remain under the Qwen Community License 1.0; framework code Apache-2.0; Strata MIT. The engine repo description tags the family "device-native foundation models".