TL;DR — real-time DE+EN voice on 8 and 12 GB
One question runs through this comparison: which open-model stack gives you a real-time German + English voice pipeline on a consumer GPU with 8 or 12 GB of VRAM?
The five findings
- Real-time starts at 8 GB. A full DE+EN stack — streaming STT (0.7 GB) + small TTS (0.9 GB) — needs under 3 GB total and runs 25–150× faster than real-time, leaving room for a quantized Whisper large-v3 alongside.
- Quantization is the biggest lever. Orpheus-3B in Q4_K_M is 6.7× faster than HF bf16 (129 vs 40 tok/s) with 3× less VRAM. Whisper int8 saves ~36% VRAM at ±5% speed.
- The framework decides, not the model. Orpheus-3B via HF bf16 = RTF 2.08 (twice slower than real-time). The same 3 B model via llama.cpp Q4 on a 12 GB card is borderline real-time.
- DE+EN dual is the real filter. Only about a third of the 29 analyzed TTS models verifiably cover both languages (9 verified, 4 derived); on the STT side, 15 of 22 do.
- FP8 is a luxury. Ampere cards (RTX 3060 and friends) have no FP8 tensor cores — and don't need them: int8 (CTranslate2/ONNX) and GGUF Q8/Q4 deliver better VRAM-per-quality for this class anyway.
Key numbers
| Analyzed | |
|---|---|
| TTS models | 29 (12 DE+EN dual, 9 verified) |
| STT / ASR models | 22 (15 DE+EN dual) |
| Consumer GPUs referenced | 22, scope here: ≤ 12 GB VRAM |
| Cheapest real-time class | 8 GB GPU (full stack) |
Sources: HF Trending (TTS/ASR), HF Open TTS & ASR leaderboards, Artificial Analysis, tts-bench, gpuwerk, OrcaBot, FunASR benchmarks, model cards — Nov 2026 snapshot.
How to read this guide
Everything below this line is data. Three symbols carry judgment throughout:
| Marker | Meaning |
|---|---|
| ✅ | Measured on consumer hardware (mostly 3060/4060/4070 class) |
| ⚠️ | Derived or estimated — from parameter count × bytes, model cards, or neighboring benchmarks. Verify before production. |
| ❌ | Does not fit the stated VRAM class / not real-time |
Definitions used: RTF = processing time per second of audio (below 1.0 = faster than real-time). TTFA = time to first audio. WER = word error rate (lower is better).
VRAM budget rule of thumb: reserve ~1.1 GB of a 12 GB card for CUDA context + desktop compositor (~0.9 GB on an 8 GB card). Everything else in this guide refers to model memory, so a "fits" row always has this overhead already deducted.
Language filter: models are flagged DE+EN dual only when verified (benchmarks/community reports, ✅) or officially documented by the model card (⚠️). EN-only models are listed separately — they are often smaller and faster, but incomplete for bilingual pipelines.
TTS — the 8–12 GB shortlist
29 open TTS models analyzed — 14 are DE+EN-capable. Sorted by usefulness in the 8–12 GB class.
| Model | License | DE+EN | Quality | VRAM | Speed & notes |
|---|---|---|---|---|---|
| Kokoro-82M · 82 M | Apache 2.0 | ✅ | Elo 1064 · top-3 EN WER | 0.9 GB | 35–100×; ONNX/GGUF, CPU-capable |
| Chatterbox + Turbo · 0.5–1.2 B | MIT | ✅ | TTS-Arena 1468 | 2.9–6.4 GB | Turbo ~75 ms, 6×; emotion knob, [laugh], 23 languages |
| Fun-CosyVoice3-0.5B · 0.5 B | Apache 2.0 | ✅ | top-3 multilingual | 4.3 GB | ~150 ms first chunk; 9 languages + 18 dialects, vLLM/TRT-LLM |
| Qwen3-TTS-1.7B · 0.6/1.7 B | Apache 2.0 | ⚠️ | Seed-TTS WER 1.24 (SOTA) | 2–4 GB | 97/101 ms first packet; GGUF/vLLM, voice design |
| Ming-3.0-tts-0.5B · 1.48 B | MIT | ⚠️ | Elo 1187 — #1 MIT | ~3 GB | cloning, 100 voices |
| Piper · ~16 M | GPL-3.0 | ✅ | WER 4% | <0.5 GB | 32× on CPU (RTF 0.04); embedded, 100+ languages |
| ZONOS3-Mini · 0.8 B MoE | Apache 2.0 | ✅ | PESQ 4.02 | ~1 GB (Q4) | cloning, 3 steering knobs |
| Pocket-TTS · 100 M | CC-BY 4.0 | ⚠️ | WER 6% | ~0.5 GB | ~6× real-time on CPU |
| XTTS-v2 · 750 M | CPML (NC) | ✅ | Elo 915 — far behind | 2.1 GB | CPU RTF 1.69; 17 languages, cloning, streaming |
| Fish S2-Pro · 4 B | NC (research) | ✅ | HF top-3 EN+multi | ~8 GB | RTF 0.195; 80 languages, 44.1 kHz |
| Voxtral 4B · ~4 B | CC-BY-NC | ⚠️ | Elo 1092 | 8–12 GB | 20 voices + cloning, vLLM |
| Bark · ~0.4 B | MIT | ✅ | WER 50% ⚠ | ~12 GB | CPU RTF 4.34; historic, unreliable text adherence |
| Fish Drama 3 · 35.9 B MoE | Commercial | ✅ | #1 open — Elo 1275 | ❌ 41.5 GB | quality reference only, datacenter-class |
| Higgs Audio V2 · 4–5.8 B | Apache (V2) | ✅ | Seed-TTS WER 1.11 | ❌ ≥24 GB | music + SFX, 100 languages |
12 GB picks: Kokoro for utility voice agents, Chatterbox Turbo for expressive German narration, CosyVoice 3 if you want the vLLM/TensorRT-LLM path. Ming and Qwen3-TTS are the best permissive-licensed models — but verify German quality first (⚠️ markers).
EN-only TTS — smaller, faster, incomplete for bilingual use
| Model | License | VRAM | Notes |
|---|---|---|---|
| F5-TTS | CC-BY-NC (weights) | 0.87 GB | flow-matching, RTF 0.039 (TRT-LLM), easy fine-tuning |
| Orpheus 3B | Apache 2.0 | 2.5 GB (Q4) | Q4 is 6.7× faster than bf16; excellent prosody, ~200 ms streaming |
| Spark-TTS | Apache 2.0 | ~1.5 GB | pitch/rate/gender control, RTF 0.136 |
| ZipVoice | Apache code, NC weights | ~0.5 GB | non-AR flow-matching, P50 98.7 ms, ONNX-INT8/TensorRT |
| Phonon 31M | Apache 2.0 | ~0.2 GB | WER 1.70%, Elo 1114 — runs in the browser (WebGPU), +50 Elo over Kokoro |
| VibeVoice 1.5B | MIT | 5.3 GB | long-form up to 15 min; the 7B needs 17.6 GB |
| Dia 1.6B | Apache 2.0 | 3.5–6.3 GB | multi-speaker dialog [S1]/[S2] — 0.2–0.5× real-time, offline only |
| IndexTTS-2.5 | bilibili license | ~3 GB | emotion/timbre separation, precise duration control |
| Sesame CSM-1B | Apache 2.0 | ~3.5 GB | dialog context, WER 18% — built for voice agents |
Proprietary reference: Eleven v4 — Elo 1339/1340, #1 of 104 (API only). GPT-4o-mini-tts (API, naturalness 4.43/5): shuts down Jan 6, 2027.
STT — streaming and batch
22 open STT models analyzed — 15 cover DE+EN. The class splits between native streaming and batch accuracy.
| Model | License | DE+EN | Accuracy | VRAM | Streaming & notes |
|---|---|---|---|---|---|
| Nemotron 3.5 ASR Streaming 0.6B | OpenMDW-1.1 | ✅ | FLEURS Ø 8.84 | 0.7 GB (q8) | ✅ 80–1120 ms chunks, 240 streams (H100), 40 locales |
| Parakeet TDT 0.6B v3 | CC-BY-4.0 | ✅ | LS 1.93 / 3.59 | 2.5 GB / 0.7 (GGUF-q8) | ⚠️ 2-s pseudo-chunks; best speed/quality tradeoff, 25 EU languages |
| Whisper large-v3-turbo | MIT | ✅ | OASR 7.83 · LS 2.10 | 1.6 / 0.8 (int8) | ⚠️ chunked; 99+ languages, GGUF/INT8, massive tooling |
| Whisper large-v3 | MIT | ✅ | LS 2.01 / 3.60 | 3.1 / 2.9 (int8) | ❌ batch; word-level timestamps, translate, LID |
| Moonshine v2 | MIT | ✅ | OASR Ø 6.65 | 0.03–0.5 GB | ✅ native streaming (cache); CPU/RPi <200 ms, ONNX |
| Qwen3-ASR-1.7B | Apache 2.0 | ✅ | OASR 5.76 · LS 1.63 | ~3.4 GB (fp16) | ❌ batch; 52 languages, forced aligner, GGUF/CoreML |
| Canary-1B-Flash | CC-BY-4.0 | ✅ | LS clean 1.48 | ~4 GB | ❌ batch; en/de/es/fr + translation, PnC |
| Canary-Qwen 2.5B | CC-BY-4.0 | ✅ | OASR 5.63 — #1 | ~8 GB | ❌ batch; ASR + summarization/QA in one model |
| Canary-1B-v2 | CC-BY-4.0 | ✅ | LS 2.18 / 3.56 | ≥6 GB | ❌ batch; 25 EU languages, ASR + translation |
| Vosk (de/en) | Apache 2.0 | ✅ | EN 5.69 / DE 9.83 | ~40 MB | ✅ zero-latency partials; Pi-class hardware |
| Phi-4-multimodal | MIT | ✅ | OASR 6.02 | ~11 GB ⚠️ | ❌ batch; audio+vision+text, speech-QA |
| MMS-1B-all | CC-BY-NC | ✅ | LS 12.63 | ~2 GB | ❌ batch; 1162 languages via adapters — fine-tune base |
| SeamlessM4T v2 | CC-BY-NC | ✅ | weaker than Whisper | ~5 GB | ⚠️ streaming separate; ASR+S2ST/S2TT/T2TT |
| Kyutai STT 2.6B | CC-BY 4.0 | ❌ EN/FR only | OASR 6.40 | 1–2.6 GB | ✅ true streaming, 0.5–2.5 s delay, semantic VAD |
| Distil-Whisper large-v3 | MIT | ❌ EN | ~1% of large-v3 | ~2.4 GB | ⚠️ 6.3× faster; long-form stable |
| SpeechBrain Streaming | Apache 2.0 | ❌ EN | LS 3.13 @ 960 ms | ~0.5 GB | ✅ 320–1280 ms configurable (RNN-T) |
| SenseVoice-Small | MIT (code) | ❌ no DE | CER 7.81 (zh) | 0.45 GB | ⚠️ pseudo-streaming; emotion + event tags, 169× GPU |
Proprietary API reference (Artificial Analysis word error rate):
| Provider | Model | AA-WER | DE | Streaming | Price |
|---|---|---|---|---|---|
| ElevenLabs | Scribe v3 | 1.9% | ✅ | ✅ | — |
| StepFun | Step-Audio 3 | 2.5% | ✅ | ❌ | $0.37/1k min (cheapest) |
| AssemblyAI | Universal-2 | 3.1% | ✅ | ✅ ~300 ms | $0.15/h |
| OpenAI | GPT-4o-Transcribe | 4.0% | ✅ | ✅ 320 ms | $6/1k min |
| Chirp 3 | 4.3% | ✅ | batch only | — | |
| Deepgram | Nova-3 | 5.2% | ✅ | ✅ sub-300 ms | $4.30/1k min |
STT picks: streaming → Nemotron 0.6B q8 (best balance) or Moonshine (smallest). Batch accuracy on 12 GB → Canary-Qwen 2.5B (8 GB, fits) or Whisper large-v3 int8 (2.9 GB). CPU-only → Moonshine / Vosk.
GPUs ≤ 12 GB — capability map
Which cards does this guide cover? Everything with 12 GB or less — a large share of Steam-hardware GPUs. Raw TFLOPS matter less here than bandwidth and quantization options, which is why a 12 GB RTX 3060 with 360 GB/s and no FP8 can still run most of the stacks below.
NVIDIA (GeForce)
| GPU | VRAM | FP16 TFLOPS | Bandwidth | Notes |
|---|---|---|---|---|
| RTX 5070 | 12 GB | 123 | 672 GB/s | Blackwell, FP8 ✅ |
| RTX 4070 Ti | 12 GB | 176 | 672 GB/s | Ada, FP8 ✅ |
| RTX 4070 | 12 GB | 116 | 504 GB/s | Ada, FP8 ✅ |
| RTX 5070 Ti class 16 GB reference | 16 GB | 177 | 896 GB/s | (out of scope, for comparison) |
| RTX 3060 | 12 GB | ~25 | 360 GB/s | Ampere: no FP8, bf16 ✅, int8/GGUF ✅ |
| RTX 5060 Ti 8G | 8 GB | 95 | 448 GB/s | Blackwell, FP8 ✅ |
| RTX 4060 Ti 8G | 8 GB | 88 | 288 GB/s | Ada, FP8 ✅ |
| RTX 4060 | 8 GB | 60 | 272 GB/s | Ada, FP8 ✅ |
| RTX 3070 / 3060 Ti | 8 GB | ~40 / ~32 | 448 / 448 GB/s | Ampere, no FP8 |
| RTX 5060 / 3060 / 3050 (8 GB) | 8 GB | 95 / ~24 / ~18 | 336–448 GB/s | entry real-time class |
| RTX 3050 6G | 6 GB | ~18 | 192 GB/s | tight but workable for small stacks |
AMD · Intel
| GPU | VRAM | Approx. compute | Bandwidth | Notes |
|---|---|---|---|---|
| RX 9070 GRE | 12 GB | ~74 | 640 GB/s | ROCm 7 (gfx1201), patchy TTS/ASR support |
| RX 7700 XT / 6750 XT | 12 GB | ~60 / ~40 | 384–432 GB/s | ROCm coverage varies |
| RX 6700 | 10 GB | ~30 | 320 GB/s | RDNA2, ROCm legacy |
| RX 7600 | 8 GB | ~42 (AMD convention) | 288 GB/s | AMD convention ≠ NVIDIA dense — not 1:1 comparable |
| Arc B580 | 12 GB | ~27 (XMX) | 456 GB/s | OpenVINO paths, best-VRAM-per-Euro |
| Arc B570 / A750 | 10 / 8 GB | ~24 | ~360–456 GB/s | OpenVINO/ONNX preferable |
Apple Silicon note: M-series with unified memory behaves differently (no separate VRAM, MLX runtimes). Whisper runs ~12× real-time on an M4 Pro and Kokoro/F5 have MLX builds — but the VRAM budgets in this guide do not transfer directly.
Why bandwidth is the 12 GB bottleneck: with 3 B-class models, generation speed tracks memory bandwidth, not TFLOPS. On a 360 GB/s card, Orpheus-3B Q4 stays at RTF ≈ 0.9–1.1 — a 4090 with ≫1000 GB/s reaches 0.64.
Speed by VRAM class — 8 GB vs 12 GB
The bar lengths are relative to the fastest entry in each column. 8 GB and 12 GB columns only — 16/24 GB classes are out of scope here.
STT speed — real-time multiples (higher = better)
| Model | VRAM fp16 / quantized | 8 GB | 12 GB |
|---|---|---|---|
| Whisper large-v3-turbo | 1.6 / 0.8 int8 | 40–56× | 46–56× |
| Parakeet TDT 0.6B v3 | 2.5 / 0.7 GGUF | 50–140× | ~150× |
| SenseVoice-Small | 0.45 | ~80–100× | ~100× |
| Whisper large-v3 | 3.1 / 2.9 int8 | int8 only | 12–23× |
| Paraformer-Large | 0.40 | ~60× | ~80× |
| Nemotron 3.5 Streaming 0.6B | ~0.7 (q8_0) | ✅ runs | ✅ + many streams |
| Fun-ASR-Nano (LLM decoder) | ~16 GB | ❌ OOM | ❌ OOM |
TTS speed — real-time multiples (higher = better)
| Model | VRAM measured | 8 GB | 12 GB |
|---|---|---|---|
| Kokoro-82M | 0.93 GB | 105 ms (4060) | 75 ms |
| Piper | <0.5 GB | CPU RTF 0.04 | CPU ok |
| F5-TTS | 0.87 GB | ~1–3× | RTF 0.15 |
| Chatterbox Turbo | 3.0–3.3 GB | ~2× | ~2.5× |
| Chatterbox (0.5–2.7 B) | 2.9–6.4 GB | Q4, RTF 1.0–1.5 | RTF ~1.0 |
| CosyVoice 3 0.5B | 4.34 GB | ~1.0–1.2× | ~1.5× |
| Orpheus-3B · llama.cpp Q4 | ~2.5 GB | bandwidth-limited | RTF ~0.9–1.1 ⚠️ |
| Orpheus-3B · HF bf16 ⚠️ | 6.7 GB | OOM-risk | RTF 2.08 — not real-time |
| XTTS-v2 | 2.1 GB | ~1.5× | ~2.5× |
| Dia / CSM | 3.5–6.3 GB | 0.3–0.6× | ~0.5× |
Where the numbers come from: measured values are the 3060/4060/4070/4090-class benchmarks; the 8 GB column is extrapolated from 4060/4060-Ti measurements. All marked rows carry ⚠️. The Orpheus row pair is the single most instructive comparison in this guide: identical model, different runtime → 6.7× speed difference.
What runs real-time where
| Class | Verdict |
|---|---|
| Every dedicated GPU (incl. 3050, RX 7600, Arc B580) | ✅ Whisper large-v3/turbo at 15–80×; CPU only ~5× |
| 8 GB VRAM | ✅ Parakeet / SenseVoice / Nemotron at 50–150× · Kokoro & Piper also on CPU — a full DE+EN stack |
| 12 GB VRAM | ✅ + Whisper large-v3, Chatterbox bf16, CosyVoice · ⚠️ Orpheus Q4 at the boundary (bandwidth) |
| Autoregressive large TTS (Dia, CSM, VibeVoice 7B) | ❌ 0.2–0.55× even on flagship cards — offline/batch only |
Ready-to-run stacks for 8 GB and 12 GB
Budget math: a 12 GB card offers ≈ 10.9 GB for models and runtime (≈1.1 GB overhead), an 8 GB card ≈ 7.0 GB (≈0.9 GB overhead). Every stack below fits with that deducted.
| # | Class | Stack (STT + TTS) | Total | Headroom | Real-time? |
|---|---|---|---|---|---|
| A | 8/12 GB | Nemotron 0.6B q8 + Kokoro ONNX | 2.8 GB | 9.2 GB | ✅ STT 80–150× ⚠️, TTS ~25–40× |
| B | 8 GB | Whisper-turbo int8 + Parakeet-tailor + Chatterbox Turbo | ~5.6 GB | ~1.4 GB ⚠️ | ✅ full stack ⚠️ |
| C | 12 GB | Whisper large-v3 int8 + Chatterbox Turbo bf16 | 7.3 GB | 4.7 GB | ✅ STT 10–14×, TTS ~2.5×, TTFA ~75 ms |
| C′ | 12 GB | same, quantized: Whisper int8 + Chatterbox Q8 | 8.0 GB | 4.0 GB | ✅ +28% TTS speed in Q4 variant |
| D | 12 GB | Parakeet v3 q8 + Orpheus-3B Q4 (EN) | 4.3 GB | 7.7 GB | ⚠️ RTF 0.9–1.1 — bandwidth-limited; add Kokoro (0.95) for DE → 5.25 GB |
| E | 12 GB | Parakeet v3 q8 + CosyVoice 3 0.5B fp16 | 6.1 GB | 5.9 GB | ✅ ~1.5×, DE+EN + 7 more languages |
| F | 12 GB | Whisper-turbo int8 + Dia 1.6B bf16 (offline) | 7.6 GB | 4.4 GB | ❌ ~0.2× — offline dialog/audiobooks only |
| G | 12 GB | Canary-1B-Flash + XTTS-v2 (batch ASR) | 8.4–9.2 GB | ~2.8 GB | ✅ ASR offline (RTFx 749 on H100); safer: Canary-Flash ≈ 4 GB |
The OOM example is the lesson: Whisper large-v3 fp16 (4.5 GB) + Chatterbox bf16 (6.4 GB) = 12.0 GB + 1.1 GB overhead → CUDA OOM. Same stack quantized (int8 + Q8_0) = 8.0 GB total with 4 GB to spare — audio quality difference: ≈ −0.00x UTMOS, practically indistinguishable.
Quantization matrix (8–12 GB class)
Values measured where unmarked; ⚠️ = derived. Ampere rule: int8 ✅ · GGUF ✅ · FP8 ❌ (FP8 needs Ada/Blackwell).
| Model | fp16/bf16 | int8 (CTr2/ONNX) | GGUF Q8_0 | GGUF Q4_K_M |
|---|---|---|---|---|
| Whisper large-v3 | 4.50 GB | 2.90 GB ✅ (−36%) | ~2.50 ⚠️ | ~2.00 ⚠️ |
| Whisper large-v3-turbo | 2.50 GB ✅ | 1.50 GB ✅ | ~1.30 ⚠️ | ~1.05 ⚠️ |
| Parakeet TDT 0.6B v3 | 1.20 GB (fp32 2.50) | ~0.65 | 0.70 ✅ q8_0 | ~0.60 ⚠️ |
| Nemotron 3.5 Streaming 0.6B | 1.20 GB | ~0.62 | 0.70 ✅ q8_0 | ~0.60 ⚠️ |
| Kokoro-82M | 0.93 GB ✅ | ~0.60 (ONNX) ← official | ~0.55 ⚠️ | ~0.42 ⚠️ |
| Chatterbox (var.) | 6.40 GB ✅ | ~3.50 ⚠️ | 4.00 ✅ Q8 | 2.90 ✅ Q4 |
| Orpheus-3B | 6.70 GB ✅ | ~3.35 ⚠️ | ~4.60 ⚠️ | 2.50 ✅ — 6.7× faster |
| CosyVoice 3 0.5B | 4.34 GB ✅ Peak | ~2.30 ⚠️ | ~2.40 ⚠️ | ~1.75 ⚠️ |
| F5-TTS | 0.87 GB ✅ | ~0.50 (ONNX) | ~0.52 ⚠️ | ~0.40 ⚠️ |
| XTTS-v2 | 2.10 GB ✅ | ~1.40 (ONNX) | ~1.50 ⚠️ | ~1.15 ⚠️ |
| Canary-1B-Flash | ~4.00 ⚠️ | ~2.40 ⚠️ | ~3.30 ⚠️ | ~1.85 ⚠️ |
Faustregel: STT belongs in int8/Q8 — nearly free. TTS decides between Q4 speed (+28%, ~0.002 UTMOS loss vs Q8) and Q8 safety. Whichever model family you pick, prefer the Turbo/Flash/Mini variant over quantizing the big one: Whisper turbo is 6× faster than large-v3 by architecture alone.
Top-5 rankings (≤ 12 GB lens)
Six curated lists, filtered for the ≤ 12 GB lens. Full rankings live in the linked leaderboards.
TTS — best quality
- Ming-3.0-tts-0.5B — Elo 1187, #1 permissive (MIT) ⚠️ DE unverified
- Fish Drama 3 — Elo 1275, #1 open overall (needs datacenter VRAM)
- Qwen3-TTS-1.7B — Seed-TTS WER 1.24 (SOTA), Apache
- Chatterbox — TTS-Arena 1468, MIT, DE+EN
- Kokoro-82M — Elo 1064, DE+EN, fastest quality-per-GB
TTS — fastest / lowest latency
- AuroraTTS-100K — 12 ms TTFA warm, 603× RTFx on 5090
- Kokoro-82M — 35–100× (4060 Ti class), CPU-capable
- Qwen3-TTS 0.6B/1.7B — 97/101 ms first packet, RTF 0.23
- ZipVoice-Distill — P50 98.7 ms, ONNX-INT8/TensorRT
- Pocket-TTS / Chatterbox-Turbo / Piper — ~6× real-time on plain CPUs
TTS — best quality/speed tradeoff
- Ming-3.0 — Elo 1187 at <3 GB, MIT
- Qwen3-TTS-1.7B — SOTA accuracy + 97 ms streaming + GGUF/vLLM
- Kokoro-82M — top-3 WER, 35–100×, 0.9 GB
- Phonon 31M — WER 1.70%, Elo 1114, browser-runnable
- VoxCPM2 — 3.3× on 4090, 30 languages, ~8 GB
STT — best accuracy
- Canary-Qwen 2.5B — OASR 5.63, CC-BY, ~8 GB (fits 12)
- Qwen3-ASR-1.7B — OASR 5.76, LS 1.63, 52 languages, Apache
- Whisper large-v3 — LS 2.01/3.60, MIT, word timestamps
- ElevenLabs Scribe v3 — AA-WER 1.9% (API, proprietary)
- Nemotron 3.5 Transcribe 30B-A4B — AA-WER 2.8% (68 GB — best open accuracy, out of VRAM scope)
STT — fastest / lowest latency
- Parakeet TDT 0.6B v3 — RTFx 3333 (A100), DE+EN, 0.7 GB quantized
- Canary-1B-Flash — >1000 RTFx
- faster-whisper int8 — 23× (large-v3) / 56× (turbo) on 4070 Ti
- Nemotron 3.5 Streaming 0.6B — 80 ms chunks, ~0.7 GB
- Moonshine Tiny / Vosk small — <200 ms on Pi & phones, native streaming
STT — best quality/speed tradeoff
- Parakeet TDT v3 — OASR 6.32 at RTFx >3300, 25 EU languages, CC-BY
- Qwen3-ASR 0.6B/1.7B — 52 languages incl. DE, Apache, 2000× throughput (0.6B)
- Whisper large-v3-turbo — OASR 7.83 at 6× the large-v3 speed, 99+ languages
- Canary-1B-Flash — LS clean 1.48 at ~4 GB
- Nemotron 3.5 Streaming — real streaming + 40 locales at 0.7 GB
Pick by scenario
| Scenario | STT | TTS | Notes |
|---|---|---|---|
| Web / voice agent, low latency | Nemotron 3.5 Streaming 0.6B (80 ms chunks) | Kokoro ONNX or Chatterbox Turbo | full real-time, 12 GB → fits twice over |
| Best quality on ≤ 12 GB | Canary-Qwen 2.5B (~8 GB) | CosyVoice 3 0.5B | batch STT + streaming TTS, one 12 GB card |
| Commercially free (MIT/Apache/CC-BY) | Whisper (MIT) + Parakeet (CC-BY) | Kokoro (Apache) + Chatterbox (MIT) | every license clear for products |
| CPU / embedded / Raspberry Pi | Moonshine Tiny / Vosk small | Piper (RTF 0.04) | no GPU at all — 32× real-time |
| Offline desktop, no cloud | faster-whisper int8 / whisper.cpp | Kokoro + Piper | 99+ languages STT, both sides CPU-GPU hybrid |
| Expressive German narration | any batch model | Chatterbox (emotion knob, [laugh]) |
23 languages, MIT licensing |
| Transcription batches (hours of audio) | Whisper-turbo int8 batch 8–16 | — | 35–60× on 12 GB ⚠️, batch 8 = sweet spot |
The safest single answer for most readers with a 12 GB card: Parakeet TDT v3 + Whisper large-v3 int8 + Kokoro-82M + Chatterbox Turbo — all four together (~6.5 GB) cover streaming, batch, utility and expressive voice, all in DE+EN, all permissive licenses.
Corrections & caveats
- ZipVoice is TTS, not ASR — the associated ASR models are icefall/k2-fsa Zipformer encoders (LS-WER 5.04).
- Leaderboard RTFx numbers (Parakeet 3380, Canary >1000) come from datacenter GPUs with high batch — on a 4060/4070 expect single-stream real-time multiples (see the class tables above).
- Deepgram Nova-3 WER is contradictory: Artificial Analysis 5.2% vs. own test 13.78% vs. AssemblyAI 6.66%.
- Chatterbox repo marked retired-research since 08/2026 — models remain on HF; OpenAI GPT-4o-mini-tts shuts down 2027-01-06.
- DE language flags with ⚠️ (Qwen3-TTS, Ming-3.0, Voxtral, Pocket-TTS) are derived from model cards, not verified against German WER benchmarks.
- All "× real-time" values for the 8 GB / 12 GB columns are extrapolated from 4060/4060-Ti/4070 measurements unless marked ✅; license and DE quality should be re-verified before production use.
- VRAM values marked ⚠️ are from parameter count × bytes or nearby hardware; measured values are listed in the text above the tables.
Sources
Leaderboards & index pages
- HF Open TTS Leaderboard · HF Open ASR Leaderboard
- Artificial Analysis TTS · Artificial Analysis STT
- HF TTS Trending · HF ASR Trending
Surveys & benchmarks
- gpuwerk — 15 open-source TTS models · OrcaBot open-weight TTS benchmark · tts-bench · Modal — open-source TTS
- Northflank — best open STT 2026 · FunASR benchmarks · x2q — Whisper backend shootout
Hardware speed references
- whisper.cpp · GigaGPU — Whisper 4090 · GigaGPU — Kokoro per GPU · MLX bandwidth · Orpheus vLLM/TRT-LLM streaming
Key model cards
- Kokoro-82M · Chatterbox · CosyVoice3 · Parakeet TDT v3 · Canary-1B-v2 · Nemotron ASR Streaming · Whisper large-v3-turbo · faster-whisper · Qwen3-ASR · Moonshine · Vosk models · ZipVoice · F5-TTS · Higgs TTS 3
Values come from public leaderboards and model cards as of Nov 2026; RTFx/WER numbers are configuration-dependent and serve as guidance. Verify license and German speech quality (⚠️ markers) before production use.