LFM3-Omni-2B

About

LFM3-Omni-2B is Liquid AI's first omni-modal model in the LFM3 family: a single end-to-end model that understands text, images, video and audio and answers in text and speech - one model that replaces a separate ASR model, vision-language model, LLM and TTS model, built for on-device use on phones, laptops and other edge hardware. It has 2.17B total parameters (2.2B class) and is maintained and trained by LiquidAI (Liquid AI).

Architecture. The checkpoint combines four parts around the LFM3 backbone: the 24-layer LFM3 tri-hybrid language model (13 gated short convolution blocks, 5 sliding-window attention layers, 6 global attention layers with grouped-query attention (GQA) 16/4 KV heads at head dim 128, partial rotary embeddings with rotary factor 0.5 on global layers, sandwich norm, tied embeddings, hidden size 2048), a 14-layer LFM3 vision encoder (convolution and bidirectional attention, 16x16 patches, 2D axial rotary embeddings) attached through a GELU projector (hidden size 4096), a causal LFM audio encoder (17-layer convolution/attention hybrid, 128 mel bins, 16 kHz) and the in-house LiquidCodec audio tokenizer whose tokens drive a 3-layer DepthFormer detokenizer (maskgit-style parallel audio-codebook decode) to render 24 kHz speech. Speech is generated with 12 codebooks (codebook size 2017), representing roughly 80 ms of audio as 12 discrete tokens; images are processed at native resolution (up to 2,048 tokens per image, each 2x2 patch block becomes one language token) and video is sampled at 2 fps with 4 to 384 frames, optionally with its own soundtrack interleaved - the model sees and hears a video at the same time via use_audio_in_video.

Usage. Input text, images, video (with or without soundtrack) and 16 kHz audio; output text, 24 kHz speech, or interleaved text and speech (generation modes text, audio, interleaved, controlled by system prompt or generation_mode). This is a non-reasoning model: enable_thinking=True is not supported and answers open with an empty think block (<think></think>); greedy decoding works well for text, while speech output uses audio_temperature=0.4 and audio_repetition_penalty=1.2. Context length is 32K tokens (input + output) with a 128,000-token text vocabulary. Text languages are the 16 LFM3 languages (English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish) and speech covers 19 languages and locales (Arabic, Danish, German, Greek, British English, American English, Finnish, French, Hebrew, Italian, Japanese, Korean, Dutch, Polish, Portuguese, Spanish, Swedish, Turkish, Chinese). It supports speech recognition (ASR), speech synthesis (TTS), speech-to-text and speech-to-speech chat, translation, image and video understanding, tool calling from spoken requests (Qwen3-style XML tool calls), and fine-tuning via Transformers SFT (omni and VQA notebooks).

Performance (all scores measured by Liquid AI, 0-100 scale except OCRBench out of 1,000). Omni benchmarks: Daily-Omni 70.09, WorldSense 47.10, AV-SpeakerBench 46.79, OmniBench (overall) 46.32, OmniBench (speech) 48.25 - all ahead of the 3B and often the 7B-class Qwen2.5-Omni - plus spoken visual QA (RealWorldQA 64.68, ERQA 39.43, EmbSpatial 67.59). Vision: MMStar 55.67, MMMU (val) 42.44, MMMU-Pro 27.28, DocVQA 92.47, POPE 89.34, OCRBench 806, RefCOCO 73.55, Video-MME (no subtitles) 57.70. Audio: English ASR WER 6.36 (Open ASR / ESB suite), multilingual ASR error 6.72 (FLEURS), English TTS UTMOSv2 3.80, VoiceBench 64.73, voice tool calling 52.20, MMAR 51.40.

Deployment. Checkpoints ship as BF16 (SGLang serves text output) and GGUF (llama.cpp, including a realtime speech server). Released October 27, 2026 (LFM3 release blog post) under the LFM Open License v1.0.