GPU Models for LLM Inference — Specs, VRAM & Prices
Browse and compare GPU hardware for LLM inference
GeForce RTX 3080
NVIDIA · Ampere
VRAM
10 GB
Memory BW
760 GB/s
FP16
119 TFLOPS
MSRP
USD 699.00
GeForce RTX 4090 D
NVIDIA · Ada Lovelace
VRAM
24 GB
Memory BW
1,008 GB/s
FP16
290.6 TFLOPS
MSRP
USD 1,835.00
GeForce GTX 1650 Super
NVIDIA · Turing
VRAM
4 GB
Memory BW
192 GB/s
FP16
— TFLOPS
MSRP
USD 159.00
GeForce RTX 4060 Ti 16GB
NVIDIA · Ada Lovelace
VRAM
16 GB
Memory BW
288 GB/s
FP16
88 TFLOPS
MSRP
USD 499.00
GeForce RTX 5080
NVIDIA · Blackwell
VRAM
16 GB
Memory BW
960 GB/s
FP16
225.1 TFLOPS
MSRP
USD 999.00
GeForce RTX 5070 Ti
NVIDIA · Blackwell
VRAM
16 GB
Memory BW
896 GB/s
FP16
177.4 TFLOPS
MSRP
USD 749.00
GeForce RTX 3070
NVIDIA · Ampere
VRAM
8 GB
Memory BW
448 GB/s
FP16
81.3 TFLOPS
MSRP
USD 499.00
GeForce RTX 5050
NVIDIA · Blackwell
VRAM
8 GB
Memory BW
320 GB/s
FP16
52.6 TFLOPS
MSRP
USD 249.00
GeForce RTX 4070 Super
NVIDIA · Ada Lovelace
VRAM
12 GB
Memory BW
504 GB/s
FP16
142 TFLOPS
MSRP
USD 599.00
GeForce RTX 3060
NVIDIA · Ampere
VRAM
12 GB
Memory BW
360 GB/s
FP16
51.2 TFLOPS
MSRP
USD 329.00
GeForce RTX 4080
NVIDIA · Ada Lovelace
VRAM
16 GB
Memory BW
716.8 GB/s
FP16
194.9 TFLOPS
MSRP
USD 1,199.00
AMD Radeon RX 7700 XT
AMD · RDNA3
VRAM
12 GB
Memory BW
432 GB/s
FP16
35.17 TFLOPS
MSRP
USD 449.00
AMD Radeon RX 9060
AMD · RDNA4
VRAM
8 GB
Memory BW
320 GB/s
FP16
21.4 TFLOPS
MSRP
USD 279.00
AMD Radeon RX 6600
AMD · RDNA2
VRAM
8 GB
Memory BW
224 GB/s
FP16
17.86 TFLOPS
MSRP
USD 329.00
AMD Radeon RX 6700
AMD · RDNA2
VRAM
10 GB
Memory BW
320 GB/s
FP16
22.58 TFLOPS
MSRP
USD 479.00
AMD Radeon RX 7900 XTX
AMD · RDNA3
VRAM
24 GB
Memory BW
960 GB/s
FP16
61.44 TFLOPS
MSRP
USD 999.00
AMD Radeon RX 6400
AMD · RDNA2
VRAM
4 GB
Memory BW
128 GB/s
FP16
7.13 TFLOPS
MSRP
USD 159.00
AMD Radeon RX 6800 XT
AMD · RDNA2
VRAM
16 GB
Memory BW
512 GB/s
FP16
41.47 TFLOPS
MSRP
USD 649.00
Apple M2 Ultra
Apple · M2
VRAM
64 GB
Memory BW
800 GB/s
FP16
54.4 TFLOPS
MSRP
USD 3,999.00