Ripple-65B Preview

About

Ripple-65B Preview is a 65B-parameter dense (non-MoE) Transformer instruction-tuned language model from Ricursive Intelligence (HuggingFace org ricursiveAI), pretrained from scratch on 12 trillion tokens of a predominantly-English and code corpus at 8K sequence length, with support for up to 1Mi (1,048,576 tokens) of context. The launch blog (ricursive.com/ripple) describes Ripple-65B Preview as the general-purpose variant of the proprietary model powering Ricursive's AI chip-design platform.

Model family and training recipe

Ripple-65B follows the Ripple v1 model ladder, derived from an initial Olmo 3 baseline with iterative improvements to pretraining scaling laws, architecture, data, and post-training; the ladder improved Compute Efficiency (CE) on code by 16x relative to the Olmo 3 ladder, increasing to 55x for the full hero run. The org reports a full-stack R&D cost of roughly $32M for developing and training the model (launch blog, incl. ladder development, data mixing, evals, synthetic data generation, and the final hero runs). The training recipe (from the card's stage table):

• Pretrain: 12T tokens, 877,500 steps, 8K seqlen, batch 1408 -> 3072 — 12T text-only predominantly-English + code corpus; pretraining kept "clean" with minimal synthetic data.
• Midtrain: 200B tokens, 8,000 steps, 64K seqlen, batch 384 — synthetic-data-heavy stage added here so the pretraining data stayed clean.
• Long Context: 6.5B tokens, 389 steps, 2M seqlen, batch 8.
• SFT: 183B tokens, 7,274 steps, 128K seqlen, batch 192.
• LC SFT: 13.5B tokens, 802 steps, 2M seqlen, batch 8.
• RL (mixed domain): 700 steps, 128K seqlen, batch 2048.

The Preview release went through deliberately light post-training with a focus on agentic coding. Ripple-65B Base Preview is the pretrain-only sibling checkpoint (no midtraining/synthetic QA data); this repo contains the BF16 instruction-tuned model.

Architecture

Ripple-65B is a thin-and-deep dense Transformer: narrow d_model (5120) with 80 total layers (config.json; FFN intermediate size 40960). Config.json swaps the Olmo 3 llama-like block for a Gemma 3N-like architecture for training stability at scale: 64 of 80 layers are sliding-window attention (SWA, window 1024) and 16 are full attention (4:1 ratio). Attention is GQA with 80 query heads over 8 KV heads, head_dim 128, deliberately overprovisioned in head count with an increased SWA-to-global ratio and reduced SWA window compensating the extra cost. Additional reasoning-oriented components added relative to Olmo 3: FFN activation clipping (ffn_clamp_threshold 20.0), gated attention (use_gated_attention: true), global NoPE (no RoPE on global layers; local SWA layers carry rope_theta 10000.0), and value residuals (use_valres: true). Vocab 100352, tie_word_embeddings: false, rms_norm_eps 1e-06, embed scale 71.55, max_position_embeddings 2,097,152, auto class RicursiveForCausalLM (model_type ricursive), safetensors in BF16.

Ripple-65B was codesigned for NVIDIA hardware: ~40% MFU throughout pretraining/posttraining with simple sharding, no custom kernels, and no quantization (BF16); serving reaches ~250 tok/s on 2xB200 (batch=8) prior to quantization or custom kernels. At 65B total parameters it fits consumer hardware such as the NVIDIA DGX Spark, and the org ships an NVFP4 checkpoint plus a colab for local development.

Ecosystem

Artifacts: BF16 instruct model (this repo), NVFP4 quantization (Ripple-65B-Preview-NVFP4), base model (Ripple-65B-Base-Preview), DFlash2 variant (Ripple-65B-Preview-DFlash2); Apache 2.0 license. Launch platform availability (via launch blog): Hugging Face, Together AI, GMI Cloud, OpenRouter, Fireworks. On HF: 5 finetunes, 43 quantizations, no official HF Inference Provider; downloads last month 5,087, 414 likes.

Benchmarks (agentic coding first)

Ripple-65B Preview scores 80.52 on Terminal-Bench 2.1 (self-reported headline; the comparison table prints 80.5 for both the self-reported and the Terminus-2 row), positioned on agentic coding above Reflection Beam 501B (80.1), Meta Muse Spark Open 816B (76.4), Thinking Machines Inkling 975B (63.8), and NVIDIA Nemotron 3 Ultra 550B (56.4). Other instruct-model scores from the card: SWE-bench Verified 76.5, SWE-bench Pro v1.0 61.42, SWE-bench Pro v2.0 Hard 79.1, DeepSWE 1.1 44.3, MCP Atlas 71.67, AutomationBench (Public) 36.4, tau3-bench banking 31.27, SciCode (Inspect AI) 39.4; knowledge/math: MMLU 89.6, MMLU-Pro 84.2, GPQA Diamond 85.2, AIME 2024 97.1 / 2025 98.3 / 2026 95.0, HMMT Feb 2026 90.5, IFBench 79.2, AA-LCR 75.0, StrongREJECT 97.5, FORTRESS Adversarial 83.8, FORTRESS Benign 95.5.

Evaluation methodology (as stated on the card): all models tested at full BF16 with maximum effort levels; table mixes official model-card scores with Ricursive's own in-house eval flow (marked italicized in the card); Terminus 2 used for SWE-bench Pro v2.0 Hard and DeepSWE 1.1 on Ripple, OpenCode otherwise unless harness specified. Replication of agentic evals is explicitly called out as unstable (e.g. TB 2.1 scores lower with official Terminus 2 at 128K + summarization), so results are a "best-effort" fair comparison.

Maintainer: Ricursive Intelligence (ricursive.com, contact info@ricursive.com). Base-model pretraining evals (MMLU 83.7, MMLU-Pro 61.5, HumanEval 70.9, GSM8K 88.5, etc.) belong to the Ripple-65B-Preview-Base artifact and are not attributed to this instruct record.

Training Data Pretrained from scratch on a 12T token, text-only predominantly-English and code corpus at 8K sequence length with minimal synthetic data kept out of pretraining; significantly more synthetic data was added during midtraining (200B tokens), followed by a light posttraining recipe (midtraining, long-context training, SFT, long-context SFT, mixed-domain RL) with a focus on agentic coding.