Guides · Updated

Build a private voice assistant on your own PC

Turn a microphone, local speech recognition, a local chat model and a local voice into a spoken assistant that stays on your computer.

Not sure which model to pull? Use the GPU model picker: enter your VRAM and RAM and get a list of Ollama models that fit.

A voice assistant is four steps: the microphone, speech-to-text, a chat model, and text-to-speech. Each of those can run on your own machine. This guide connects them and lists the settings that keep the audio there. The speech tools themselves are in Local speech-to-text and text-to-speech.

What stays on the PC

  • Speech-to-text: a local Whisper model, as in the speech guide. Some browsers’ built-in dictation sends audio to the browser maker. Use the local option.
  • The chat model: a model you have pulled onto the PC. In the Ollama app, a tag ending in :cloud or -cloud runs on Ollama’s servers. Leave those tags unused, then check the chat still answers with the network unplugged. The full list of off switches is in What leaves your PC?.
  • The spoken reply: Piper or Kokoro on your machine. Open WebUI’s default text-to-speech uses browser voices, and some of those are online. Point it at a local server.
  • The apps: Ollama and the chat window listen on your computer. Ollama’s API has no login, so leave it on localhost. Port-forwarding publishes that API to the internet.

Once the models are downloaded, the conversation works with the network switched off. That is the practical test. The full walkthrough is in the privacy checklist.

The simple version: talk to Open WebUI

  1. Install Ollama on Windows, Mac or Linux.
  2. Pull a model that fits your memory. Spoken chat works better with a model that starts answering promptly. A reasoning model that writes a long chain of thought before the first word makes an awkward conversation, unless the app lets you turn thinking off. The GPU picker lists models that fit.
  3. Install Open WebUI and keep it bound to your own computer.
  4. In Admin Panel → Settings → Audio, set speech-to-text to the local Whisper option. For text-to-speech, use a local Piper or Kokoro server. Model sizes and install notes are in the speech guide.
  5. Use the microphone button, or Standard call mode, for a back-and-forth conversation. Leave Call mode on Standard: Realtime sends the audio to OpenAI or Azure.

Add a short instruction such as “reply in one or two spoken sentences.” Long answers take longer to generate, and they are tiring to listen to.

What each piece needs

PieceHardware
Whisper (speech-to-text)Smaller sizes run on a CPU. A GPU is faster. Sizes are in the speech guide.
Chat modelThe heavy part. It needs the same memory as any other local chat.
Piper or Kokoro (text-to-speech)Piper is comfortable on a CPU. Kokoro runs on a CPU and is faster with a GPU.

On an 8 GB graphics card, run Whisper on the CPU so the chat model can use the GPU. A quiet room and a reasonable microphone improve accuracy more than stepping up a Whisper size.

On a laptop, plug in for a long conversation. See Running local AI on a laptop.

If a live conversation is too slow on your hardware, transcribe a recording first and then ask the model about the transcript. That uses the same tools, just not in real time.

A wake word

The microphone button is the private default: audio is captured while you hold it.

A wake word keeps the microphone open, waiting for a phrase. Home Assistant’s Assist pipeline can do that on your own hardware: Whisper for speech-to-text, Piper for the voice, its local wake-word option, and the Ollama integration as the conversation agent. Follow Home Assistant’s current voice documentation for the add-ons, because the screens change. If Assist is pointed at a cloud conversation agent, the questions leave your network.

Tell people in the room that a wake word is listening. Transcripts of speech are often more sensitive than typed notes. Store them with the same care as in the privacy checklist, including full-disk encryption.

Before it can do things

Answering questions is the right first version. Sending messages, running commands or controlling devices means it can act on a misheard word. Set that up with the limits in Running local AI agents safely.

Using this for client calls or a small office? See Local AI for a small business: a practical starter.

Sources: Open WebUI docs, Whisper, Home Assistant voice, Ollama integration for Home Assistant.

Get the Local AI weekly

New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.

Subscribe free →

Lila Forge Kits

Local-LLM App Starter

A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.

kits.lilaforge.com →
VibeEngine

Best local LLM tools, ranked

Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.

vibeengine.lilaforge.com →