Guides · Updated

Local AI for a small business: a practical starter

A first local AI setup for a small team: one computer, one chat app, your own documents, and the privacy, licence and review checks before anyone relies on it.

Not sure which model to pull? Use the GPU model picker: enter your VRAM and RAM and get a list of Ollama models that fit.

Local AI earns its place in a small business when the text is private: client notes, quotes, a staff handbook, draft emails. The useful version is small. One computer, one app, one job you can check by hand. The reasons to bother are in Why run AI locally?

Start with one job

A good first jobA poor first job
Drafting an email or a job post from notes you provideA chatbot on your public website
Summarising a meeting recorded on your own machineLetting the model send email or edit files
Answering staff questions from a handbook you uploadedReplacing an accountant, a lawyer or a clinician
Turning a messy brief into a checklist a person will editFine-tuning a model before a plain one has been tried

Pick the job you actually do every week. If the answers are useful there, add a second job later.

The starter setup

  1. Use a computer you already have if it can run a small model. What that looks like: Local AI on a budget PC. If you are buying a quiet machine for the office, read Choosing a mini PC for local AI and check the GPU picker.
  2. Install one app. One person can use Ollama or LM Studio (how they differ). Several people sharing the same model: Open WebUI, which has user accounts. The first account is the administrator. After the people who need access have accounts, turn off further sign-ups in the admin settings.
  3. Read the model licence on its model card before you use it for paid work. Some models allow commercial use. Others restrict it, or allow it only under extra terms. Where to look: Ollama models explained.
  4. Add one document set. The handbook, the price list, or the policies people already ask about. How upload and search work: Chat with your documents locally.
  5. Write a house rule in one sentence: client files, staff records and unreleased work go into the local app, and they stay out of cloud chatbots.

Sharing it inside the office

Ollama has no password of its own. Keep it on localhost, or reach the office machine through a VPN such as Tailscale. Port-forwarding it to the internet publishes an API anyone can call. Open WebUI can sit in front with its own accounts. The steps are in Local AI for privacy, step by step. The team case is step 9.

A few matching habits:

  • Put the computer on the staff network, not on guest Wi-Fi.
  • Turn on full-disk encryption. Chats and uploaded files sit on the drive.
  • Decide who owns backups. A cloud backup of the app’s data folder copies those files off the premises.
  • Agree who installs updates and removes old models. The housekeeping is in Keeping local models up to date and tidy.

A person checks the answer

These models write fluent text that can still be wrong. For anything a customer will see, and for anything about money, health or the law, someone checks it against the source before it goes out. A document search helps the model stick to your files, and it can still mix passages up. Treat the draft as a draft.

Sending, deleting, browsing or running commands is a bigger step. Read Running local AI agents safely before you turn that on.

Meetings and phone audio

Transcribe recordings on the same computer with Whisper, then ask the model about the transcript. Live spoken chat is in Build a private voice assistant on your own PC. The speech tools are in Local speech-to-text and text-to-speech.

What you pay

The apps in this guide cost nothing to download, and there is no per-message fee. The costs are the computer and the electricity. A PC you already own is enough to try it. Buying a machine for the office makes more sense when several people will use it, or when the work cannot go to a cloud service. How the electricity compares with a monthly plan, in NZ dollars and US dollars, is in What does running AI at home cost?. The same trade-off is sketched in Why run AI locally?.

What this does not decide for you

Keeping processing in the office can make a confidentiality promise easier to keep, and it can make questions about where files are processed easier to answer. It does not, by itself, meet a contract or a regulation. Read the obligation you actually have, and match the setup to it: who can log in, where backups go, and how long chats are kept.

Sources: Ollama FAQ, Open WebUI docs.

Get the Local AI weekly

New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.

Subscribe free →

Lila Forge Kits

Local-LLM App Starter

A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.

kits.lilaforge.com →
VibeEngine

Best local LLM tools, ranked

Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.

vibeengine.lilaforge.com →