Guides · Updated
Local AI for a small business: a practical starter
A first local AI setup for a small team: one computer, one chat app, your own documents, and the privacy, licence and review checks before anyone relies on it.
Local AI earns its place in a small business when the text is private: client notes, quotes, a staff handbook, draft emails. The useful version is small. One computer, one app, one job you can check by hand. The reasons to bother are in Why run AI locally?
Start with one job
| A good first job | A poor first job |
|---|---|
| Drafting an email or a job post from notes you provide | A chatbot on your public website |
| Summarising a meeting recorded on your own machine | Letting the model send email or edit files |
| Answering staff questions from a handbook you uploaded | Replacing an accountant, a lawyer or a clinician |
| Turning a messy brief into a checklist a person will edit | Fine-tuning a model before a plain one has been tried |
Pick the job you actually do every week. If the answers are useful there, add a second job later.
The starter setup
- Use a computer you already have if it can run a small model. What that looks like: Local AI on a budget PC. If you are buying a quiet machine for the office, read Choosing a mini PC for local AI and check the GPU picker.
- Install one app. One person can use Ollama or LM Studio (how they differ). Several people sharing the same model: Open WebUI, which has user accounts. The first account is the administrator. After the people who need access have accounts, turn off further sign-ups in the admin settings.
- Read the model licence on its model card before you use it for paid work. Some models allow commercial use. Others restrict it, or allow it only under extra terms. Where to look: Ollama models explained.
- Add one document set. The handbook, the price list, or the policies people already ask about. How upload and search work: Chat with your documents locally.
- Write a house rule in one sentence: client files, staff records and unreleased work go into the local app, and they stay out of cloud chatbots.
Sharing it inside the office
Ollama has no password of its own. Keep it on localhost, or reach the office machine through a VPN such as Tailscale. Port-forwarding it to the internet publishes an API anyone can call. Open WebUI can sit in front with its own accounts. The steps are in Local AI for privacy, step by step. The team case is step 9.
A few matching habits:
- Put the computer on the staff network, not on guest Wi-Fi.
- Turn on full-disk encryption. Chats and uploaded files sit on the drive.
- Decide who owns backups. A cloud backup of the app’s data folder copies those files off the premises.
- Agree who installs updates and removes old models. The housekeeping is in Keeping local models up to date and tidy.
A person checks the answer
These models write fluent text that can still be wrong. For anything a customer will see, and for anything about money, health or the law, someone checks it against the source before it goes out. A document search helps the model stick to your files, and it can still mix passages up. Treat the draft as a draft.
Sending, deleting, browsing or running commands is a bigger step. Read Running local AI agents safely before you turn that on.
Meetings and phone audio
Transcribe recordings on the same computer with Whisper, then ask the model about the transcript. Live spoken chat is in Build a private voice assistant on your own PC. The speech tools are in Local speech-to-text and text-to-speech.
What you pay
The apps in this guide cost nothing to download, and there is no per-message fee. The costs are the computer and the electricity. A PC you already own is enough to try it. Buying a machine for the office makes more sense when several people will use it, or when the work cannot go to a cloud service. How the electricity compares with a monthly plan, in NZ dollars and US dollars, is in What does running AI at home cost?. The same trade-off is sketched in Why run AI locally?.
What this does not decide for you
Keeping processing in the office can make a confidentiality promise easier to keep, and it can make questions about where files are processed easier to answer. It does not, by itself, meet a contract or a regulation. Read the obligation you actually have, and match the setup to it: who can log in, where backups go, and how long chats are kept.
Sources: Ollama FAQ, Open WebUI docs.
Get the Local AI weekly
New models worth running, what fits which GPU, and setup fixes. One short email a week, free. Unsubscribe any time.
Local-LLM App Starter
A tested starter for building your own app on Ollama: chat, RAG and a GPU-fit checker. Skip the setup hours.
kits.lilaforge.com → VibeEngineBest local LLM tools, ranked
Ollama, LM Studio, Open WebUI and llama.cpp scored from public evidence, with top picks.
vibeengine.lilaforge.com →