Your smart home knows when you wake up, how much energy you use, and whether you left the garage door open. But here's the uncomfortable question: where does that data actually go? For most people using Alexa, Google Home, or Siri, the answer is someone else's cloud server. Every voice command, every temperature reading, every "what's on my calendar today?" gets processed remotely.
That model works fine—until it doesn't. Privacy advocates have long warned about the risks, but the trade-off seemed unavoidable. You needed cloud AI to get genuinely smart home automation.
That trade-off is dissolving. Open-source large language models (LLMs) have improved to the point where they can run on hardware you already own. Combined with Home Assistant—the open-source home automation platform that runs on a Raspberry Pi or an old PC—you can build a fully local AI assistant that understands natural language, controls your devices, and never sends a single packet to a third-party server.
This article walks you through the entire process: the technology behind it, the hardware you'll need, step-by-step setup, customization, and real-world examples. By the end, you'll have a private, self-hosted AI assistant that answers to you and no one else.
Ollama is an open-source tool that runs large language models locally on your own hardware. Instead of sending prompts to OpenAI's servers or Anthropic's API, Ollama downloads model weights directly to your machine and runs inference on your CPU or GPU. Launched in 2023, it has been downloaded over 10 million times since.
The tool provides a simple command-line interface, a RESTful API, and a library of over 300 open-source models. You type ollama run llama3, and within minutes you're chatting with a model running entirely on your hardware. No account creation. No API keys. No usage limits.
Home Assistant is a free, open-source home automation platform that runs locally on your network. It integrates with over 1,000 different devices and services—from Zigbee sensors and Z-Wave locks to Philips Hue lights and Nest thermostats. Unlike cloud-dependent hubs like SmartThings or Wink, Home Assistant continues to function even when your internet goes down.
Its core philosophy is local control and privacy. All your device data, automation history, and configuration files stay on your own hardware.
Home Assistant has a built-in "conversation" integration that lets you interact with your home using natural language. By default, it uses basic pattern matching—you say "turn off the lights," and it looks for keywords. With Ollama as the conversation agent, Home Assistant sends your natural language commands to the local LLM, which interprets them and triggers the appropriate device actions.
The integration works through Ollama's API endpoint (default port 11434). Home Assistant sends a prompt, the LLM processes it, and returns either a text response or a function call that executes a specific automation.
When you use Alexa or Google Home, every command is recorded, transcribed, and stored on company servers. With a local setup, all conversation logs, device states, and sensor data remain on your private network. This matters if you're using your assistant for sensitive tasks like checking bank balances, managing security cameras, or controlling door locks.
Cloud AI requires a round trip to a remote server. Even with fast internet, you're looking at 1–2 seconds of latency per request. A local LLM on a modern GPU can respond in under 500 milliseconds. More importantly, a local assistant works when your internet goes down—an advantage if you rely on voice commands for critical functions like locking doors or turning off appliances.
Cloud AI services charge per token. A heavy smart home user making hundreds of requests daily could spend $20–50 per month. Ollama is completely free. The only cost is electricity and hardware depreciation.
Let's be honest: local models aren't as capable as GPT-4 or Claude 3.5. They struggle with complex reasoning, have smaller knowledge bases, and can hallucinate more frequently. For home automation—which is mostly short, structured commands—this limitation rarely matters. But if you're expecting your local assistant to draft legal documents or solve advanced math problems, you'll be disappointed.
Key Takeaway: Self-hosted AI gives you privacy, speed, and zero per-request costs. The trade-off is raw intelligence. For smart home tasks, that's a fair exchange.
The hardware floor depends on which model you choose. For the smallest usable models (like Phi-3 Mini or Gemma 2B), you need:
Running on CPU alone is possible but slow. A 7B parameter model on a modern CPU generates roughly 5–10 tokens per second. That's fine for text responses but sluggish for voice interactions.
A GPU changes everything. An NVIDIA RTX 3060 (12GB VRAM) runs Llama 3 8B at 40–60 tokens per second. An RTX 4090 can handle larger models like Llama 3 70B at usable speeds.
If you're using a Raspberry Pi or a mini-PC without a discrete GPU, stick to models with 3B parameters or fewer. They'll be less capable but still functional for basic commands.
| Model | Parameters | Minimum RAM | Best For |
|---|---|---|---|
| Llama 3 | 8B | 8GB | General purpose, good instruction following |
| Mistral | 7B | 8GB | Fast inference, solid reasoning |
| Gemma | 2B | 4GB | Low-powered devices like Raspberry Pi |
| Phi-3 Mini | 3.8B | 4GB | Compact, surprisingly capable |
| Llama 3 | 70B | 32GB | Complex queries, needs serious hardware |
Key Takeaway: Match the model to your hardware. An 8B model on 8GB RAM works but will be slow. A 2B model on a Raspberry Pi is responsive but limited. There's no free lunch.
On Linux or macOS, run:
curl -fsSL https://ollama.com/install.sh | sh
Windows users download the installer from ollama.com. Once installed, verify with:
ollama --version
ollama run llama3
This downloads the model (about 4.7GB) and starts an interactive session. To keep Ollama running as a background service, use:
ollama serve
The API will be available at http://localhost:11434.
The easiest method is Home Assistant OS, which you flash to an SD card for a Raspberry Pi, or install as a virtual machine. For existing servers, the Docker installation is straightforward:
docker run -d --name homeassistant --restart=unless-stopped \
-v /path/to/config:/config \
-v /etc/localtime:/etc/localtime:ro \
-p 8123:8123 \
ghcr.io/home-assistant/home-assistant:stable
Open your Home Assistant instance (usually http://your-server:8123). Navigate to Settings → Devices & Services → Add Integration. Search for "Ollama." Enter the URL of your Ollama server (e.g., http://192.168.1.50:11434), select your model, and save.
The native integration is solid, but the HACS (Home Assistant Community Store) version adds features like function calling support and better error handling. To install:
In Settings → Voice Assistant, create a new assistant. Set the conversation agent to "Ollama." You can now test it in the Home Assistant dashboard by typing "turn off the living room lights."
The system prompt tells the LLM how to behave. In the Ollama integration settings, you can add something like:
"You are Jarvis, a helpful home assistant. Respond concisely. When the user gives a command, acknowledge it and explain what you're doing. If you can't perform a task, say so directly."
You can make it formal, sarcastic, or completely utilitarian. The prompt shapes everything.
The HACS Ollama integration supports function calling. This lets the LLM invoke Home Assistant services directly. For example, if a user says "I'm cold," the LLM can call the climate service to raise the thermostat by 2 degrees—without a pre-built automation.
You can also use Ollama inside Home Assistant automations. Create a script that sends sensor data to the LLM and asks for a decision:
automation:
- alias: "Smart Energy Decision"
trigger:
platform: time_pattern
hours: "/1"
action:
- service: ollama.generate
data:
prompt: "Current energy price is {{ states('sensor.energy_price') }}. Should I charge the EV now? Reply yes or no."
For a fully voice-controlled assistant, pair Ollama with:
Both integrate natively with Home Assistant and run entirely on your hardware.
Key Takeaway: The system prompt is the personality dial. Function calling is the action dial. Together, they turn a generic LLM into a purpose-built home assistant.
A Raspberry Pi 5 with 8GB RAM runs Llama 3 8B. When you say "Good morning," the AI:
All responses come through local text-to-speech. Zero cloud involvement.
A user with an RTX 3060 runs Mistral 7B. They've set up an automation that queries the LLM hourly with current energy prices, outdoor temperature, and home occupancy. The AI decides whether to pre-heat the house before peak pricing hours and sends a notification explaining its reasoning.
When a motion sensor triggers at 2 AM, Home Assistant sends the event to Ollama: "Motion detected at the back door at 2:03 AM. The user is asleep. What should I do?" The LLM responds with instructions: activate exterior lights, send an alert to the user's phone, and start recording from the security camera.
A user with an RTX 4090 runs Llama 3 70B. They can ask questions like, "What's the most energy-efficient way to heat my home today, given that electricity costs $0.32/kWh from 4-9 PM?" The model processes real-time data and delivers a nuanced, context-aware recommendation.
"Connection refused" error: The Ollama server isn't reachable. Check that it's running (ollama serve), the port is open, and your firewall allows traffic on port 11434.
Slow responses: Your model is too large for your hardware. Switch to a smaller model or reduce context length in the integration settings.
Model doesn't understand device names: Your system prompt needs to include a list of your devices and their entity IDs. Add them explicitly.
Ollama exposes metrics at http://localhost:11434/api/ps. For real-time monitoring, use htop or a tool like Grafana with the Prometheus exporter.
Ollama handles concurrent requests, but each request uses memory. If multiple users are querying simultaneously, you'll need more RAM or a GPU with larger VRAM. For heavy use, consider running Ollama on a dedicated server rather than the same machine as Home Assistant.
The core benefit of this setup is data locality. But verify that your Home Assistant instance isn't accidentally sending data elsewhere. Disable any cloud integrations you don't need, and check your network traffic periodically.
Home Assistant has multi-user support. Create separate user accounts for family members with appropriate permissions. The conversation agent uses the permissions of the user who initiates the conversation, so a child's account won't be able to unlock doors.
Key Takeaway: Local AI removes the cloud risk but introduces your own security responsibilities. A properly configured self-hosted setup is more private than any cloud service—if you configure it correctly.
Open-source models are improving rapidly. Llama 3.1, Mistral's latest releases, and Qwen 2.5 all show that local models are closing the gap with cloud offerings. The community is also developing smaller, more efficient architectures that run on edge devices.
Home Assistant's voice assistant pipeline is actively under development, with better wake-word detection and more natural conversation flows. Ollama continues to add model support and performance optimizations, including experimental multimodal models that can process images.
Building a self-hosted AI assistant with Ollama and Home Assistant is not a weekend project for everyone. It requires hardware, configuration, and a willingness to troubleshoot. But the payoff is real: a private, fast, and cost-effective assistant that answers to you.
The right choice depends on your priorities. If you're comfortable with cloud services and don't mind the privacy trade-offs, existing solutions work fine. If you value data ownership and want an assistant that works even when the internet doesn't, this setup is worth the effort.
Start small. Install Ollama on a machine you already have. Download a small model. Connect it to Home Assistant. Add a voice pipeline. You can always upgrade hardware and models later.
At minimum, 8GB of RAM and a decent CPU. A Raspberry Pi 5 with 8GB is the entry point. For comfortable performance, a mini-PC with 16GB RAM or a desktop with a dedicated GPU (NVIDIA RTX 3060 or better) is recommended.
Yes. Once the model is downloaded and Home Assistant is configured, everything runs locally. The only exception is integrations that require cloud services (e.g., weather APIs, some device brands).
Go to Settings → Devices & Services → Add Integration → search for "Ollama." Enter your Ollama server's URL and select your model. For the enhanced version with function calling, install via HACS.
Any device that Home Assistant can control, the AI can control—provided you've set up the integration correctly and the device entities are exposed to the conversation agent.
Local models are smaller, so they have less general knowledge and weaker reasoning. They can also hallucinate more. For structured home automation commands, they're more than adequate. For open-ended conversation, they'll disappoint.
Match model size to available RAM. A 7B model needs roughly 8GB of RAM. An 8B model needs 8–12GB. A 70B model needs 32GB+. If in doubt, start with a smaller model and upgrade if you need more capability.
Absolutely. The system prompt in the Ollama integration settings defines the assistant's personality, tone, and behavioral constraints. You can make it formal, playful, minimalist, or anything in between.
Yes, if configured correctly. Data stays on your network. Just ensure your network is secure, don't expose Ollama's API to the internet, and use strong passwords for Home Assistant.
You can run Ollama on CPU. Expect 5–10 tokens per second with a 7B model—usable but not snappy. Stick to 2B–4B models for better responsiveness.
Yes. Ollama's API is RESTful and works with any application that can make HTTP requests. You can connect it to Node-RED, custom scripts, web apps, or even use it as a backend for your own chatbot.
Ready to take control of your smart home AI? Start by exploring the Ollama model library and installing Home Assistant today. Join the community forums to share your setup and learn from others.