Raon-1 mini

About

Raon-1 mini is a 30B-A4B Mixture-of-Experts (MoE) language model designed and trained from scratch by KRAFTON AI. As part of the Raon open model family, it is released as a preview release under the Apache 2.0 license. It demonstrates strong capabilities in Korean conversation and agentic tasks, and ships with a built-in thinking mode that the maintainers recommend keeping enabled.

Architecture

Raon-1 mini uses a hybrid Mixture-of-Experts (MoE) architecture with 25 layers — 18 linear-attention layers and 7 full-attention layers:

• Raon Delta Attention (RDA): a linear-attention layer that decouples the erase and write gates of Kimi Delta Attention (KDA), improving long-context performance over KDA. Raon Delta Attention layers carry a fixed-size state (no KV cache).
• Gated MLA: only the 7 MLA (multi-head latent attention) layers keep a KV cache, about 8 KiB per token in BF16. Layers follow a 3:1 RDA : Gated MLA layout.
• Raon-mHC: reduces the computational cost of DeepSeek's manifold-constrained hyper-connections (mHC), which speeds up training; used with 4 residual streams.
• Sparse MoE: 6 of 128 routed experts plus 1 shared expert are active per token, so each token uses 4.3B of the 28.8B total parameters (expert intermediate size 1,088; the first layer is a dense FFN with intermediate size 12,032).
• NoPE (no positional encoding), SiTU-GLU activation with sigmoid MoE routing, hidden size 2,560, 128 experts, vocabulary of 259,840 with an untied LM head.
• Long context: natively supports up to 262,144 tokens of context.

Languages

Trained on Korean, English, Japanese, Chinese, Spanish, French, and Thai. Demonstrates strong Korean performance and solid English capabilities.

Agentic use and speculative decoding

Tool calling is supported for agentic workflows, with documented integration for Hermes Agent, Pi, and OpenCode through an OpenAI-compatible API. For inference acceleration, a DFlash2 draft model is provided for speculative decoding — on a single NVIDIA B300 it delivers up to 2.18× higher output throughput than autoregressive decoding.

Quantization and deployment

Raon-1 mini is available in MXFP8 (base checkpoint, dynamic activation quantization) and NVFP4 formats — NVFP4 enables inference on a single NVIDIA RTX 5090 (32 GB VRAM). GGUF checkpoints for llama.cpp and MLX checkpoints for Apple silicon are also provided. Inference is supported with Transformers, vLLM (KRAFTON fork), llama.cpp (KRAFTON branch), and MLX.

Preview benchmark highlights

Evaluator-run preview results (judge gpt-5.6-luna), leading within the MoE comparison group on math/science reasoning, instruction following and search, and Korean conversation: AIME 2026 93.3, GPQA Diamond 84.3, HMMT Feb 2026 84.1, MMLU Pro 76.5, IFBench 84.2, LiveCodeBench v6 81.6, SWE-Verified 70.8 (mini-SWE-agent harness), BrowseComp Plus 55.7, and an internal Chatbot Korean score of 92.9. As a preview release, specs and results may change before the final version.

Maintained by KRAFTON AI (krafton.ai); a free hosted playground is available on the Raon console.

Training Data Trained from scratch by KRAFTON AI on multilingual data spanning Korean, English, Japanese, Chinese, Spanish, French, and Thai; built-in thinking mode, and KRAFTON's proprietary training datasets and methodologies are not distributed with the open release.