Guides · Updated

Local AI FAQ

Straight answers to the questions beginners ask most about running AI on their own computer, with links to the guide that goes deeper.

What does "running AI locally" actually mean?

The AI model runs on your own computer instead of on a company's servers. You download the model once, and an app such as Ollama or LM Studio runs it on your graphics card, your Mac's chip or your CPU. Your prompts and files are processed on your machine, and once a model is downloaded it works without an internet connection.

The trade-off is that you can only run models your hardware can hold. More in Why run AI locally?

Do I need an expensive graphics card?

No. Almost any reasonably recent PC can run small models on its CPU, just more slowly. A graphics card with more VRAM lets you run bigger, more capable models much faster, and an Apple Silicon Mac with plenty of memory works well too.

See Local AI on a budget PC for what ordinary hardware can do, and which upgrades make the biggest difference.

How much VRAM or memory do I need?

It depends on the model. As a rule of thumb, the model's download size plus a gigabyte or two for the conversation has to fit in your graphics card's VRAM (or a Mac's unified memory) to run at full speed. With the usual 4-bit versions, an 8 GB card handles 7-9B models, 12 GB handles 12-14B, 16 GB handles around 20-24B, and 24 GB handles 27-32B.

The GPU model picker checks your exact setup, and VRAM and quantisation explained shows where those numbers come from.

Is a local model as good as ChatGPT or other cloud AI?

For many everyday jobs, such as drafting, summarising, explaining code and answering questions about your own documents, a good local model is genuinely useful. But the models that fit on a home PC are much smaller than the biggest cloud models, which still lead on hard reasoning and complex tasks.

Many people use local models for private or routine work and a cloud service for the hardest problems. More in Why run AI locally?

Is it really free?

The popular apps (Ollama, LM Studio and Open WebUI) cost nothing to use, and many models are free to download. There are no per-message fees or usage limits from those apps. The money you add is electricity while a model is running, and the hardware if you still have to buy it.

A worked example in NZ dollars and US dollars, with the power price as a figure you can change, is in What does running AI at home cost?

Some models also have licence terms for commercial use, so check before using one for business.

Should I start with Ollama or LM Studio?

Both are free and run the same kinds of open models. LM Studio is a polished desktop app, the easiest choice if you want to browse models and chat without a terminal. Ollama runs in the background, is open source, and is what most coding assistants and chat interfaces connect to. Plenty of people install both.

Full comparison: LM Studio vs Ollama. Ready to install? Start with Ollama on Windows, Mac or Linux.

Which model should I download first?

Pick the biggest model for your job that fits in your memory with a little room to spare. The GPU model picker suggests models for your exact VRAM, RAM and task.

To understand what names like 9b or 30b-a3b mean, and which kind of model suits chat, coding, reasoning, images or documents, read Ollama models explained.

Is my data really private, and does it work offline?

The model itself runs on your machine, so your prompts and files aren't sent to an AI provider, and once it's downloaded it works offline. But the app around it matters:

  • Some apps send usage data or sync chats to the cloud. Check their settings.
  • Ollama also offers cloud-hosted models, whose names end in cloud. Those run on Ollama's servers, not your PC.
  • Some tools, such as Cursor, route requests through their own servers even when the model is on your PC.

The full checklist, including how to prove nothing leaves your PC: Local AI for privacy, step by step. The documented off switch for each app is in What leaves your PC?

Why is it so slow?

Usually because the model doesn't fit fully in your graphics card's memory, so part of it runs from much slower system RAM. Run ollama ps: if it shows a CPU/GPU split or 100% CPU, try a smaller model, a shorter context, or closing other apps that use VRAM.

Two other common causes: the first reply after a break is slower because the model has to load from disk again, and reasoning models spend time "thinking" before they answer. More fixes in When it doesn't work.

Can I run it on a Mac or a laptop?

Yes. Apple Silicon Macs (M1 and later) are some of the easiest machines for local AI: there are no GPU drivers to install, and the GPU can use most of the Mac's memory. Intel Macs run models on the CPU only, which is slow.

Laptops work too, but a laptop graphics card often has less VRAM than the desktop card with the same name, and speed depends on its power limits and cooling. See Local AI on a Mac and Running local AI on a laptop.

Does it work with AMD or Intel graphics cards?

Yes, with more caveats than NVIDIA. Ollama runs on supported AMD Radeon cards through ROCm, and on many other AMD cards and Intel Arc cards through Vulkan. NVIDIA is still the smoothest path, especially for image generation.

Which cards work, and how to set them up: Running local AI on AMD, Intel Arc and mini PCs.

Can it answer questions about my own documents?

Yes. This is called RAG (retrieval-augmented generation): the app finds the most relevant parts of your files and gives them to the model along with your question. Open WebUI, LM Studio and AnythingLLM can do it without any code, and everything stays on your machine.

Setup: Chat with your documents locally. If answers ignore parts of a long document, the context length is probably set too short; see When it doesn't work.

Can I use it as a coding assistant?

Yes. Extensions such as Continue connect VS Code or JetBrains to models running in Ollama, for chat, edits and tab autocomplete, and your code stays on your machine. Local models won't match the best cloud models on hard tasks, but they're useful for explanations, small edits and tests.

Cursor can't connect to a model on your PC directly, so its requests go through Cursor's servers. Setup and details: Set up a free local coding assistant.

Can I make images with it?

Yes, but with different tools from chat. Image models such as Stable Diffusion and FLUX run in apps like ComfyUI, InvokeAI, or Draw Things on a Mac, not in Ollama. A graphics card matters even more for images than for chat; on a CPU alone it's painfully slow.

Getting started: Generating images locally.

Are downloaded models safe, and can I use them for work?

Stick to trusted sources: the Ollama library, the original publisher on Hugging Face, or well-known uploaders. For image models, prefer .safetensors files over older .ckpt and .pt files, which can run code when loaded. Add-ons such as ComfyUI custom nodes are ordinary code with full access to your PC, so install only well-known ones.

Licences differ between models, and some restrict commercial use, so read the licence on the model card before using a model for business. A first setup for a small team: Local AI for a small business. More in Ollama models explained and Generating images locally.

Came across a word you don't recognise? The local AI glossary explains VRAM, quantisation, GGUF, tokens, RAG and more in plain English.

Get the Local AI weekly

New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.

Subscribe free →

Lila Forge Kits

Local-LLM App Starter

A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.

kits.lilaforge.com →
VibeEngine

Best local LLM tools, ranked

Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.

vibeengine.lilaforge.com →