Guides · Updated
Local AI on a budget PC (no big GPU)
What an ordinary desktop or office PC can run with no graphics card or only a small one, the settings that help most, and the cheapest upgrades that make a real difference.
You don’t need an expensive graphics card to try local AI. Almost any reasonably recent PC can run small models on its CPU, and the right settings make a big difference. Expect smaller models and slower replies than on a gaming GPU, but the same privacy and no running costs beyond electricity.
What decides speed without a GPU
- How much RAM you have. Without a graphics card, the model loads into system RAM, alongside Windows and your browser. RAM size decides which models fit.
- How fast that RAM is. Generating text means reading the whole model for every word, so memory bandwidth sets the pace. That’s why it’s slower than a GPU, whose memory is many times faster. The details: VRAM and quantisation explained.
- One stick or two. Two matching RAM sticks (dual channel) give roughly twice the bandwidth of a single stick. On Windows, Task Manager → Performance → Memory shows Slots used. If it says 1, adding a matching stick is a cheap speed-up.
- The CPU. More cores and a newer CPU help most when the model reads your prompt, which matters for long documents.
What your PC can run
| Your PC | Realistic models | Try first |
|---|---|---|
| No GPU, 8 GB RAM | 1-3B | gemma3:1b, llama3.2:3b |
| No GPU, 16 GB RAM | 3-4B comfortably, 7-9B slowly | qwen3.5:4b, phi4-mini, qwen3:4b |
| No GPU, 32 GB RAM | 7-9B, plus mixture-of-experts models | qwen3.5:9b, gpt-oss:20b, qwen3:30b |
| Small GPU, 4 GB | 3B fully on the GPU | llama3.2:3b |
| Small GPU, 6 GB | 4B fully on the GPU, bigger models split with RAM | qwen3.5:4b, qwen3:4b |
The 32 GB trick: mixture-of-experts models such as gpt-oss:20b and qwen3:30b need memory for the whole model but only use a few billion parameters for each word. On a CPU with 32 GB of RAM they can be far more usable than their size suggests, and much smarter than a 4B model. Ollama models explained covers how this works.
For your exact machine, choose No GPU / CPU only in the GPU picker.
Get the most out of it
- Install Ollama or LM Studio. Ollama runs fine on the CPU with no extra setup: see the guides for Windows and Linux. LM Studio shows whether a model fits your memory before you download it (comparison).
- Start small. Run
ollama run qwen3.5:4b, then try the next size up if replies feel quick enough. - Close memory hogs. Browser tabs can use several gigabytes of RAM that the model could be using.
- Keep chats short. Long conversations and big documents use more memory and make the CPU spend longer reading before each reply. Start a new chat when you change topic.
- One model at a time.
ollama psshows what’s loaded;ollama stop <model>unloads it. - Skip thinking for quick questions. Reasoning models write out their working before answering, which takes a long time on a CPU. Use an all-rounder, or turn thinking off where your app allows it.
Integrated graphics and NPUs
- Integrated graphics share the same system RAM as the CPU, so they hit the same bandwidth limit. Ollama and LM Studio can use some integrated GPUs through Vulkan, but the gain over the CPU is often small.
- NPUs (“AI PC” chips) mostly aren’t used by popular local chat tools like Ollama. Don’t pay extra for one to run chat models.
Cheapest upgrades that make a real difference
- More RAM. Going to 32 GB as a matching pair is often the cheapest upgrade, and it unlocks the mixture-of-experts models above. Check which type (DDR4 or DDR5) and how much your motherboard supports first.
- A 12 GB graphics card. This is the biggest jump: models that crawl on a CPU become fast. A used RTX 3060 12GB or a new Intel Arc B580 are the usual budget choices. See 12GB GPU picks and what a 12GB card runs. If you’re buying for AI, skip 8 GB cards; the extra VRAM is worth it.
- An SSD, if the PC still has a hard drive. Models load much faster, though replies don’t get any quicker once a model is loaded.
Before you buy a graphics card, check that your power supply has enough wattage and the right power connectors, and that the card fits your case. Intel Arc cards need Resizable BAR switched on for good performance, which some older PCs don’t support. AMD and Intel cards run through ROCm or Vulkan; see AMD, Intel Arc and mini PCs.
Have an older graphics card already? Even a 4-6 GB card helps with small models. Very old cards may not be supported, so check Ollama’s hardware support page.
What else runs well on modest hardware
- Speech-to-text and text-to-speech: Whisper’s smaller models and Piper run well on a CPU. See Local speech-to-text and text-to-speech.
- Chatting with your documents: a small model plus
nomic-embed-textworks on a CPU. See local RAG with Ollama. - Code autocomplete:
qwen2.5-coder:1.5bis tiny and fast. See the local coding assistant guide. - Image generation really does need a GPU. A 4-6 GB card can run Stable Diffusion 1.5. See Generating images locally.
Buying a small new machine rather than upgrading this one? See Choosing a mini PC for local AI. On a laptop instead? See Running local AI on a laptop. If something isn’t working, see When it doesn’t work.
Sources: Ollama hardware support, LM Studio docs.
Get the Local AI weekly
New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.
Local-LLM App Starter
A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.
kits.lilaforge.com → VibeEngineBest local LLM tools, ranked
Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.
vibeengine.lilaforge.com →