Hardware

24GB+ GPUs for local AI

Runs the best local models: Qwen3.6 27B, Gemma 4 31B and the 30-35B mixture-of-experts models, fully on the GPU.

Models that run well on this tier

qwen3.6:27b

27B · Q4_K_M · ~18.5 GB

Newest dense Qwen; top 24GB all-rounder.

qwen3.6:27b-coding

27B (coding tune) · Q4_K_M · ~19.5 GB

Coding-tuned Qwen3.6, best local coder on 24GB.

gemma4:31b

31B · Q4_K_M · ~21.5 GB

Dense flagship Gemma 4 for 24GB cards.

gemma4:26b

26B MoE (~4B active) · Q4_K_M · ~19.5 GB

Mixture-of-experts: big-model quality, small-model speed.

qwen3-coder:30b

30B MoE (3B active) · Q4_K_M · ~20.5 GB

Agentic coding MoE; fast even with partial offload.

deepseek-r1:32b

32B · Q4_K_M · ~21.5 GB

Reasoning distill for 24GB.

Check your exact setup in the GPU picker → · Read the full guide →

Hardware picks

NVIDIA GeForce RTX 3090 (24GB, used)

24GB GDDR6X · bandwidth 936 GB/s

The local-AI community favourite: 24GB and fast memory at used prices. Check the seller and the card's thermals.

Check price

NVIDIA GeForce RTX 4090 (24GB)

24GB GDDR6X · bandwidth 1008 GB/s

A bit faster than a 3090 with better efficiency, at a much higher price.

Check price

AMD Radeon RX 7900 XTX (24GB)

24GB GDDR6 · bandwidth 960 GB/s

24GB for less than NVIDIA. Supported by ROCm on both Linux and Windows in Ollama.

Check price

NVIDIA GeForce RTX 5090 (32GB)

32GB GDDR7 · bandwidth 1792 GB/s

The fastest consumer card, with 32GB for bigger context windows. Expensive and power hungry.

Check price
Our advice: A used 24GB card is the best upgrade most enthusiasts can make. 70B models still need 48GB+, so for those look at two cards or a large unified-memory machine.

We don't list prices because they change daily. Bandwidth figures are manufacturer specs. Disclosure: "Check price" links may be affiliate links (rel="sponsored"); we may earn a commission at no cost to you.

Get the Local AI weekly

New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.

Lila Forge Kits

Local-LLM App Starter

A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.

kits.lilaforge.com → VibeEngine

Best local LLM tools, ranked

Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.

vibeengine.lilaforge.com →