LFM3-1.7B

About

LFM3-1.7B is a text model for local agents and specialized applications and the small text flagship of Liquid AI's LFM3 family, built on the new tri-hybrid architecture. In 1.73B parameters it packs a 24-layer stack of 13 gated short convolution blocks, 5 sliding-window attention layers and 6 global attention layers; the global layers use grouped-query attention (GQA) with 16 attention heads over 4 KV heads at head dim 128 and partial rotary embeddings (rotary factor 0.5, RoPE theta 1,000,000), sliding-window layers use RoPE theta 10,000 with a 256-token sliding window, and the stack uses sandwich norm plus tied word embeddings (config.json: hidden size 2048, FFN intermediate size 7680, dtype bfloat16, model type lfm3).

The model runs with a 256K-token context window (input + output) and a 128,000-token vocabulary, and is maintained and trained by LiquidAI (Liquid AI). It speaks 16 languages: English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian and Polish. Reasoning is enabled by default via the <think> block of the ChatML template (enable_thinking=False can disable traces but remains inconsistent); tool calls are emitted as XML and the chat template supports deferred tools via tool search (defer_loading=True, tool_reference parts), so large tool catalogs can be searched and injected on demand.

Training. Pre-training builds LFM3-1.7B-Base on 22T tokens in two stages: main pre-training on web and synthetic data (about 15T tokens) that adds more scientific papers, repository-level code, math and a wider range of knowledge sources than LFM2, followed by 3T tokens of code, math and science at 4K context; mid-training then uses curated, higher-quality data with a large share of agentic traces and tool calls for the last 3.5T tokens at 32K context, and two context extensions (500B tokens to 128K, 50B tokens to 256K). Post-training has four stages: supervised fine-tuning (SFT) on 1.75T tokens covering chat, reasoning, tool calling and agentic traces; mixed reinforcement learning (RL) over many domains (math, code, SQL, structured outputs) with a length penalty for concision; multi-domain on-policy distillation (MOPD), where six specialist RL teachers (code, math, tool use, long context, agentic work, enterprise workflows) are distilled back into a single model; and finally agentic RL with tasks from web browsing to office work executed inside real agent harnesses such as Hermes Agent, Pi and OpenClaw.

Performance. Liquid AI states that LFM3-1.7B "can power agentic applications that used to need 9B parameters": it scores 45.60 on LiveClaw, 38.42 on WebArena-Verified, 41.29 on AutomationBench (public), 71.39 on BFCL V4, 80.36 on ToolSandbox and 43.40 on SWE-bench Verified (OpenCode agent) - beating the much larger Qwen3.5-9B on average (49.37 vs 47.24 over the whole table) and every same-size competitor. Coding and instruction following are strong for its size: 73.46 on LiveCodeBench v6, 96.66 on IFEval, 73.03 on IFBench and 93.72 on IFStruct. On-device it is equally aggressive: at 32K context it prefills at least 2.5x faster and decodes at least 1.8x faster than every compared model on the AMD Ryzen AI Max+ 395, Apple M5 Max and iPhone 17 Pro, and at 4-bit precision it fits in about 1 GB of memory. Served at 1,024 concurrent requests it produces 3.6 billion tokens per day on a single NVIDIA B300.

Usage. Recommended sampling parameters: temperature 0.1, top_k 50, repetition penalty 1.1 for general chat, tool use, agents and coding; temperature 1.0, top_p 0.95, top_k 20 for math and STEM. Checkpoints ship as BF16, FP8, NVFP4, GGUF, MLX 4-bit and ONNX for vLLM, SGLang, Transformers, llama.cpp, Ollama, LM Studio, MLX and ONNX Runtime. It is designed for local agents, tool calling, structured outputs and domain-specific fine-tuning (SFT, DPO, GRPO and CPT notebooks via TRL); not recommended for agentic coding or complex multi-step reasoning because of its size - use retrieval or search for knowledge-heavy tasks. Released October 27, 2026 (LFM3 release blog post) under the LFM Open License v1.0.

Training Data Pre-trained on 22T tokens total: web and synthetic data (~15T), then code, math, and science (3T at 4K context), mid-training on curated agentic data (3T at 32K context), plus 128K (500B tokens) and 256K (50B tokens) context extensions. Post-training in four stages: SFT (1.75T tokens), mixed RL across math/code/SQL/structured outputs, multi-domain on-policy distillation (MOPD) from six specialist RL teachers, and agentic RL in real agent harnesses.