Fit for GeForce RTX 3090

24GB VRAM · NVIDIA · General · INT4

✓

Fits on 1 GPU

(12 models)
1
Bonsai 2 27B — GGUF
Prism ML · 27.4B · INT4

Score

0.85

tok/s (est.)

45.9

Community tok/s

—

VRAM (est.)

15.4GB

License

Apache License 2.0

2
Gemma 3 27B PT
Google DeepMind · 27.0B · INT4

Score

0.84

tok/s (est.)

46.5

Community tok/s

—

VRAM (est.)

15.2GB

License

Gemma Terms of Use

3
Xing4.0-29B-A4B
China Telecom AI · 31.2B · MoE · INT4

Score

0.82

tok/s (est.)

90.3

Community tok/s

—

VRAM (est.)

16.5GB

License

Apache License 2.0

4
Qwen3.5-27B
Qwen Team (Alibaba Cloud) · 27.0B · INT4

Score

0.77

tok/s (est.)

46.5

Community tok/s

—

VRAM (est.)

15.2GB

License

Apache License 2.0

5
Qwen3.8-27B
Qwen Team (Alibaba Cloud) · 27.0B · INT4

Score

0.76

tok/s (est.)

46.5

Community tok/s

—

VRAM (est.)

15.2GB

License

Apache License 2.0

6
Clef
Cloudflare · 27.0B · INT4

Score

0.71

tok/s (est.)

46.5

Community tok/s

—

VRAM (est.)

15.2GB

License

Apache License 2.0

7
Qwen-AgentWorld-35B-A3B
Qwen Team (Alibaba Cloud) · 35.0B · MoE · INT4

Score

0.7

tok/s (est.)

117.2

Community tok/s

—

VRAM (est.)

18.5GB

License

Apache License 2.0

8
Ornith-1.5-35B-A3B
Ornith AI · 36.0B · MoE · INT4

Score

0.69

tok/s (est.)

117.2

Community tok/s

—

VRAM (est.)

19.0GB

License

MIT License

9
Qwen3.6-35B-A3B
Qwen Team (Alibaba Cloud) · 35.0B · MoE · INT4

Score

0.67

tok/s (est.)

117.2

Community tok/s

—

VRAM (est.)

18.5GB

License

Apache License 2.0

10
Spark-X2.5-4B
SparkLLM (XHToken) · 4.0B · INT4

Score

0.65

tok/s (est.)

253.0

Community tok/s

—

VRAM (est.)

2.3GB

License

Apache License 2.0

11
Qwen3.5-35B-A3B
Qwen Team (Alibaba Cloud) · 35.0B · MoE · INT4

Score

0.64

tok/s (est.)

117.2

Community tok/s

—

VRAM (est.)

18.5GB

License

Apache License 2.0

12
Gemma 3 12B PT
Google DeepMind · 12.0B · INT4

Score

0.64

tok/s (est.)

99.4

Community tok/s

—

VRAM (est.)

6.8GB

License

Gemma Terms of Use

⚠

Needs 2+ GPUs

(12 models)
1
NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB
NVIDIA · 74.0B · int4 · 2× GPU

Score

11.37

tok/s (est.)

81.0

Community tok/s

—

VRAM/GPU (est.)

19.6GB

License

OpenMDW License Agreement v1.1

2
Muse-Spark-Open-816B-A42B
Meta Superintelligence Lab · 816.0B · int4 · 22× GPU

Score

0.91

tok/s (est.)

173.7

Community tok/s

—

VRAM/GPU (est.)

19.5GB

License

Muse Model License

3
DeepSeek-V4.1-Flash
DeepSeek · 763.0B · int4 · 20× GPU

Score

0.89

tok/s (est.)

340.7

Community tok/s

—

VRAM/GPU (est.)

20.1GB

License

MIT License

4
Qwen3.8-Flash-Next
Qwen Team (Alibaba) · 125.0B · int4 · 4× GPU

Score

0.87

tok/s (est.)

212.1

Community tok/s

—

VRAM/GPU (est.)

16.5GB

License

Qwen Community License 1.0

5
GLM-5.3-Flash
Zhipu AI · 320.0B · int4 · 9× GPU

Score

0.87

tok/s (est.)

167.0

Community tok/s

—

VRAM/GPU (est.)

18.7GB

License

MIT License

6
Kolibri-1
Aleph Alpha · 78.1B · int4 · 3× GPU

Score

0.85

tok/s (est.)

261.0

Community tok/s

—

VRAM/GPU (est.)

13.7GB

License

Apache License 2.0

7
DeepSeek-V4-Pro-0813
DeepSeek · 1.7T · int4 · 45× GPU

Score

0.83

tok/s (est.)

272.5

Community tok/s

—

VRAM/GPU (est.)

19.9GB

License

MIT License

8
Qwen3-Next-80B-A3B-Thinking
Qwen · 80.0B · int4 · 3× GPU

Score

0.82

tok/s (est.)

290.4

Community tok/s

—

VRAM/GPU (est.)

14.0GB

License

Apache License 2.0

9
Qwen3.5-122B-A10B
Qwen Team (Alibaba Cloud) · 122.0B · int4 · 4× GPU

Score

0.82

tok/s (est.)

137.8

Community tok/s

—

VRAM/GPU (est.)

16.1GB

License

Apache License 2.0

10
GPT-OSS-120B
OpenAI · 116.8B · int4 · 4× GPU

Score

0.78

tok/s (est.)

240.2

Community tok/s

—

VRAM/GPU (est.)

15.4GB

License

Apache License 2.0

11
GLM-5.3
Zhipu AI · 753.3B · int4 · 20× GPU

Score

0.75

tok/s (est.)

156.9

Community tok/s

—

VRAM/GPU (est.)

19.9GB

License

GLM-5.3 License

12
Hy4 preview
Tencent · 770.0B · int4 · 21× GPU

Score

0.74

tok/s (est.)

146.3

Community tok/s

—

VRAM/GPU (est.)

19.3GB

License

Apache License 2.0