Guides · Updated
When it doesn't work: troubleshooting local AI
The model is too big, replies crawl, the GPU sits idle, a download stalls, a port is taken, or the chat app can't see the model. What you'll see, the likely cause, and the fix.
Most local AI trouble is one of six things: the model doesn’t fit in memory, replies are slow, the GPU isn’t doing the work, a download never finishes, a port is already taken, or the chat app is looking in the wrong place. The fixes below are for Ollama, LM Studio, and the other apps these guides use (Open WebUI, Continue and ComfyUI).
Two Ollama commands answer a lot of the questions on their own:
ollama -v # is the server running, and which version?
ollama ps # what's loaded, and is it on the GPU?
- The model is too big for your memory
- Replies are very slow
- The GPU is not being used
- A download is stuck
- The port is already in use
- The chat app can’t see the model
1. The model is too big for your memory
What you’ll see. The model never starts a chat. Ollama’s message looks like model requires more system memory (X GiB) than is available (Y GiB). LM Studio shows, before you download, that the file is unlikely to fit. ComfyUI stops with an out-of-memory error.
Likely cause. The model file, plus extra memory for the conversation, is bigger than the RAM and VRAM the app can use together. A long context makes this worse. Coding tools sometimes ask for a very large context on their own.
Fix.
- Ollama. Pick a smaller model, or a smaller tag of the same family. Unload anything you aren’t using with
ollama stop <model>(Ollama can keep more than one model loaded). Close other apps that use a lot of memory. If a coding tool requested a huge context, lower it as shown in the coding assistant guide. - LM Studio. On the model’s download page, choose a smaller quantisation. The app shows whether that file is likely to fit before you download it. Lowering GPU offload can let a too-big model load by putting some layers in system RAM, but replies then become very slow (problem 2).
- ComfyUI. The image guide covers the memory flags: start the portable or manual version with
--lowvram, or--novramas a last resort. A chat model left loaded in Ollama is using the same VRAM;ollama stop <model>frees it.
Which size actually fits is covered in VRAM and quantisation explained. The GPU picker lists Ollama models for a given card.
2. Replies are very slow
What you’ll see. The model does answer, but each word takes a long time. In Ollama, ollama ps shows a split such as 48%/52% CPU/GPU, or 100% CPU.
Likely cause. Part of the model (or the whole model) is running from system RAM, which is much slower than VRAM. A very long context has the same effect, because the conversation’s memory has to fit too. Two slower-but-normal cases: the first reply after a break, while the model loads from disk (see below), and a reasoning model spending time thinking before it writes the answer.
Fix.
- Ollama. Use a smaller model that fits fully, so
ollama psreads100% GPU. Shorten the context (see the chat forgets earlier messages). Close browsers, games and video apps that hold VRAM. If you have to split a model between GPU and RAM, mixture-of-experts models cope better than dense ones of the same size. Why that is: VRAM and quantisation explained. - LM Studio. Move GPU offload up to the highest value that still fits, and lower the context length in the same load panel. A smaller quantisation uses less memory and leaves more room for the GPU.
- Laptops and ordinary PCs. Power limits, heat and a small GPU make the same model slower than on a desktop. See Running local AI on a laptop and Local AI on a budget PC.
3. The GPU is not being used
What you’ll see. Replies are slow and the graphics card stays near idle. ollama ps shows 100% CPU. In LM Studio, GPU offload is set low, so most layers stay on the CPU. ComfyUI was started with its CPU script.
Likely cause. The app didn’t detect a supported GPU, the GPU driver is missing or old, or the app was told to use the CPU.
Fix.
- Ollama. Update the GPU driver. Minimum versions for NVIDIA and AMD are in the install guides for Windows and Linux. On NVIDIA,
nvidia-smishould list the card. The server log records GPU detection at startup:%LOCALAPPDATA%\Ollama\server.logon Windows,~/.ollama/logs/server.logon Mac,journalctl -e -u ollamaon Linux. AMD and Intel cards run through ROCm or Vulkan, and not every card is supported: AMD, Intel Arc and mini PCs. On Linux, the NVIDIA GPU can disappear after sleep;sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm, then restart Ollama, or reboot. A Docker container needs--gpus=alland the NVIDIA Container Toolkit on the host. - LM Studio. In the model load settings, raise GPU offload. On a Mac, LM Studio can use Apple’s MLX engine as well as llama.cpp; see LM Studio vs Ollama and Local AI on a Mac.
- ComfyUI. The Windows portable build has
run_nvidia_gpu.batfor an NVIDIA card andrun_cpu.batfor machines without a supported GPU. The image guide has the install paths for Mac, AMD and Intel as well.
4. A download is stuck
What you’ll see. The progress bar stops, the pull ends with an error, or the model name is rejected. Ollama says pull model manifest: file does not exist when the name or tag is wrong.
Likely cause. A mistyped name, a dropped connection, a proxy the app can’t see, a model the installed app is too old to load, or a full disk.
Fix.
- Ollama. Check the spelling against the model’s Tags tab on ollama.com/library. Run
ollama pull <model>again; an interrupted pull resumes rather than starting over. If you’re behind a proxy, setHTTPS_PROXYfor the Ollama server (the place the app runs), not only in the terminal you typed the command in. A brand-new model that will not load usually needs a newer Ollama. The Windows and Mac apps update themselves; on Linux, re-run the install script from the Linux guide. - LM Studio. Start the download again from the same model page. If the model on Hugging Face asks you to accept terms before it will download, do that in the browser and try again. LM Studio’s docs cover adding a Hugging Face access token for those models.
- Disk space. Ollama models live in
~/.ollama/models(Windows:%HOMEPATH%\.ollama\models; the Linux service:/usr/share/ollama/.ollama/models).ollama listshows the size of each one, andollama rm <model>deletes one. To store them on another drive, setOLLAMA_MODELS, with steps in the Windows and Linux guides. On Linux the new folder has to be writable by theollamauser:sudo chown -R ollama:ollama /data/ollama. Other apps keep their own copies. See Keeping local models up to date and tidy.
5. The port is already in use
What you’ll see. The server will not start. Ollama says bind: address already in use on Linux and Mac, or Only one usage of each socket address on Windows. Docker says the address is already allocated. The chat page never opens.
Likely cause. That program is already running, so a second copy cannot take the same port. Something else on the PC can own the port too.
Fix.
- Ollama (port 11434). You don’t need
ollama servewhen the app or the Linux service is already running. Useollama run. To restart with new settings, quit Ollama from the tray or menu bar and open it again, or runsudo systemctl restart ollamaon Linux. To see what owns the port:netstat -ano | findstr 11434on Windows, orsudo lsof -i :11434on Linux and Mac. To move Ollama:OLLAMA_HOST=127.0.0.1:11500. Any app that connects to it has to use that same port. - LM Studio (port 1234). The local server uses port 1234 unless you changed it (LM Studio vs Ollama). Stop the copy that’s already serving, or set a different port in LM Studio’s local server panel and point your chat app at the new one.
- Open WebUI. The chat guide publishes the Docker install on port 3000 (
127.0.0.1:3000:8080) and the pip install on port 8080. If Docker says the port is allocated, stop the other container, or change only the first number in that mapping. - ComfyUI (port 8188). The portable and manual installs in the image guide open at
http://127.0.0.1:8188. For the manual install, ComfyUI accepts another port with--port, for examplepython main.py --port 8189.
6. The chat app can’t see the model
What you’ll see. The model dropdown is empty, the app says it can’t connect, or it doesn’t recognise the model name. Ollama’s own client says something like could not connect to ollama app, is it running? (the wording varies by version).
Likely cause. The server isn’t running, the chat app is using the wrong address or port, or that model was never downloaded.
Fix.
- Ollama on this PC. Open Ollama from the Start menu or Applications and check the tray or menu bar. On Linux:
sudo systemctl start ollama, thensystemctl status ollama. Or runollama servein its own terminal. Chat apps on the same machine usehttp://localhost:11434.ollama listshows the names you can actually request. If you setOLLAMA_HOSTto a custom address, the terminal needs the same variable, because theollamacommand uses it to find the server. - Open WebUI. In Admin Panel → Settings → Connections, the Ollama address is
http://host.docker.internal:11434for the Docker install andhttp://localhost:11434for the pip install. Step by step: Running a local AI chat. On Linux, a container reaches the host only if Ollama listens beyond localhost (next point) and the container was started with--add-host=host.docker.internal:host-gateway, which the chat guide includes. - Continue. The
modelvalue in the config from the coding assistant guide has to matchollama listexactly, tag included, and Ollama has to be running. - Another device, or a Linux container. Set
OLLAMA_HOST=0.0.0.0:11434, restart Ollama, and allow port 11434 through the firewall. Ollama has no login of its own, so only do this on a trusted network or over a VPN, and never port-forward it to the internet. More in the Linux guide and Local AI for privacy, step by step. - A browser extension. Allow its origin with
OLLAMA_ORIGINS(for examplechrome-extension://*). Only add origins you trust. - LM Studio. Its own chat lists models that have finished downloading. A different app sees them only while LM Studio’s local server is started and a model is loaded. The default address is
http://localhost:1234; OpenAI-compatible clients usehttp://localhost:1234/v1. The headless server islms daemon up, noted in LM Studio vs Ollama.
The chat forgets earlier messages
Every model has a context window, and Ollama’s documented default is 4,096 tokens, which is much smaller than most models allow. When a chat or a document goes past the limit, the oldest part is dropped, usually with no error in the chat.
- In an
ollama runsession:/set parameter num_ctx 16384 - For everything: set
OLLAMA_CONTEXT_LENGTH=16384and restart Ollama, or use the context length setting in the Ollama app on Windows and Mac. LM Studio has its own context length control in the model load panel. - Per request through Ollama’s API: add
"options": {"num_ctx": 16384}.
A longer context uses more memory and can push the model off the GPU (problem 2). How that memory is spent, including KV-cache quantisation: VRAM and quantisation explained.
The first reply after a break is slow
Ollama unloads a model after it has been idle for 5 minutes by default, so the next request loads it from disk again.
- Keep it loaded longer with
OLLAMA_KEEP_ALIVE=1h, or-1to keep it loaded until Ollama restarts. Apps can also send"keep_alive"with each API request. - A loaded model holds its VRAM. Free it with
ollama stop <model>. - Loading is much quicker from an SSD than from a hard drive or a USB disk.
Settings changes do nothing?
Ollama reads environment variables when the server starts. They have to be set where the server runs, not only in the terminal where you typed a command:
- Windows: user environment variables, then quit Ollama from the tray and start it again (steps).
- Mac:
launchctl setenv NAME value, then quit and reopen Ollama (steps). - Linux:
sudo systemctl edit ollama, thensudo systemctl daemon-reload && sudo systemctl restart ollama(steps).
Still stuck?
- Read the server log (locations in problem 3). For more detail, restart with
OLLAMA_DEBUG=1. - Update the app. New model formats often need a recent release.
- Search Ollama’s GitHub issues for the exact error text.
Not sure the model itself is the right size? See Ollama models explained.
Sources: Ollama FAQ, Ollama troubleshooting, Ollama hardware support, LM Studio docs, Open WebUI docs, ComfyUI docs.
Get the Local AI weekly
New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.
Local-LLM App Starter
A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.
kits.lilaforge.com → VibeEngineBest local LLM tools, ranked
Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.
vibeengine.lilaforge.com →