Filters

-
-
-
Reset

GPU Models for LLM Inference — Specs, VRAM & Prices

Browse and compare GPU hardware for LLM inference

Inference Calculator

GeForce RTX 5090

NVIDIA · Blackwell

VRAM

32 GB

Memory BW

1,792 GB/s

FP16

419.2 TFLOPS

MSRP

USD 1,999.00

runs ~53 models (INT4) View Details

GeForce RTX 4080 Super

NVIDIA · Ada Lovelace

VRAM

16 GB

Memory BW

736 GB/s

FP16

225.8 TFLOPS

MSRP

USD 999.00

runs ~28 models (INT4) View Details

GeForce RTX 4070 Ti Super

NVIDIA · Ada Lovelace

VRAM

16 GB

Memory BW

672 GB/s

FP16

197.7 TFLOPS

MSRP

USD 799.00

runs ~28 models (INT4) View Details

GeForce RTX 3090

NVIDIA · Ampere

VRAM

24 GB

Memory BW

936 GB/s

FP16

142 TFLOPS

MSRP

USD 1,499.00

runs ~52 models (INT4) View Details

GeForce RTX 4090 D

NVIDIA · Ada Lovelace

VRAM

24 GB

Memory BW

1,008 GB/s

FP16

290.6 TFLOPS

MSRP

USD 1,835.00

runs ~52 models (INT4) View Details

GeForce RTX 4060 Ti 16GB

NVIDIA · Ada Lovelace

VRAM

16 GB

Memory BW

288 GB/s

FP16

88 TFLOPS

MSRP

USD 499.00

runs ~28 models (INT4) View Details

GeForce RTX 5080

NVIDIA · Blackwell

VRAM

16 GB

Memory BW

960 GB/s

FP16

225.1 TFLOPS

MSRP

USD 999.00

runs ~28 models (INT4) View Details

GeForce RTX 5070 Ti

NVIDIA · Blackwell

VRAM

16 GB

Memory BW

896 GB/s

FP16

177.4 TFLOPS

MSRP

USD 749.00

runs ~28 models (INT4) View Details

GeForce RTX 4080

NVIDIA · Ada Lovelace

VRAM

16 GB

Memory BW

716.8 GB/s

FP16

194.9 TFLOPS

MSRP

USD 1,199.00

runs ~28 models (INT4) View Details

AMD Radeon RX 7900 XTX

AMD · RDNA3

VRAM

24 GB

Memory BW

960 GB/s

FP16

61.44 TFLOPS

MSRP

USD 999.00

runs ~52 models (INT4) View Details

AMD Radeon RX 6800 XT

AMD · RDNA2

VRAM

16 GB

Memory BW

512 GB/s

FP16

41.47 TFLOPS

MSRP

USD 649.00

runs ~28 models (INT4) View Details

Apple M2 Ultra

Apple · M2

VRAM

64 GB

Memory BW

800 GB/s

FP16

54.4 TFLOPS

MSRP

USD 3,999.00

runs ~57 models (INT4) View Details

Apple M2 Max

Apple · M2

VRAM

32 GB

Memory BW

400 GB/s

FP16

27.2 TFLOPS

MSRP

USD 1,999.00

runs ~53 models (INT4) View Details

AMD Radeon RX 9070 XT

AMD · RDNA4

VRAM

16 GB

Memory BW

640 GB/s

FP16

48.7 TFLOPS

MSRP

USD 599.00

runs ~28 models (INT4) View Details

AMD Radeon RX 7900 GRE

AMD · RDNA3

VRAM

16 GB

Memory BW

576 GB/s

FP16

45.98 TFLOPS

MSRP

USD 549.00

runs ~28 models (INT4) View Details

Apple M2 Pro

Apple · M2

VRAM

16 GB

Memory BW

200 GB/s

FP16

13.6 TFLOPS

MSRP

USD 1,299.00

runs ~28 models (INT4) View Details

AMD Radeon RX 6900 XT

AMD · RDNA2

VRAM

16 GB

Memory BW

512 GB/s

FP16

46.08 TFLOPS

MSRP

USD 999.00

runs ~28 models (INT4) View Details

AMD Radeon RX 9060 XT 16GB

AMD · RDNA4

VRAM

16 GB

Memory BW

320 GB/s

FP16

25.6 TFLOPS

MSRP

USD 349.00

runs ~28 models (INT4) View Details

AMD Radeon RX 9070

AMD · RDNA4

VRAM

16 GB

Memory BW

640 GB/s

FP16

36.1 TFLOPS

MSRP

USD 549.00

runs ~28 models (INT4) View Details

Apple M1 Pro

Apple · M1

VRAM

16 GB

Memory BW

200 GB/s

FP16

10.6 TFLOPS

MSRP

USD 1,999.00

runs ~28 models (INT4) View Details
Previous Page 1 of 2 Next