// Edge AI · Raspberry Pi 5
01How it works
Each stage runs locally. After the initial setup, the Pi needs no internet — it listens, transcribes, reasons and answers entirely on the edge.
02Booting up
The assistant runs as a systemd unit, so it comes up on its own when the board powers on. This is the real journal from a cold boot — sudo journalctl -u localmind -f:
$ sudo journalctl -u localmind -f 21:28:13 [W:onnxruntime] GetGpuDevices — failed to detect devices under "/sys/class/drm/card0" 21:29:24 UserWarning: 'CUDAExecutionProvider' is not in available provider names. Available: 'AzureExecutionProvider, CPUExecutionProvider' 21:29:25 ⏳ Carregando Whisper... 21:29:25 ✅ Whisper pronto 21:29:25 ⏳ Carregando Wake Word... 21:29:25 ✅ Wake Word pronto — diga 'Hey Jarvis' para ativar 21:29:25 🟢 Assistente iniciado! Aguardando sua voz... 21:29:25 👂 Ouvindo... 21:29:25 🎙 Voz detectada, gravando...
Those two warnings at the top are the point, not a problem. ONNX Runtime looks for a GPU, finds no /sys/class/drm/card0, and falls back to CPUExecutionProvider. Every model in this pipeline is running on the Pi’s four Cortex-A76 cores — no accelerator, no offloading.
The board is headless on the LAN and advertises itself over mDNS, so there is no IP address to memorise — from any machine on the network:
03Hardware
⚠️ The Pi 5 has no 3.5mm jack — audio goes out over USB or Bluetooth.
04The build, end to end
Every step below is a real screenshot from the build — no reconstructions. Total time from a blank SD card to a talking assistant: about four hours, most of it spent downloading models.
4.1Flash the OS
Raspberry Pi Imager writes Raspberry Pi OS 64-bit to the card. The customisation panel is doing real work here — hostname, user, Wi-Fi and SSH are all baked in before first boot, which is what makes the board reachable headless as rasp-ai.local without ever attaching a keyboard.
4.2First SSH, then update everything
Straight into ssh rasp-ai@rasp-ai.local and a full upgrade. Worth noting what scrolls past: arm64 packages from Debian trixie and a linux-image-6.18.34-rpt-rpi-2712 kernel — 2712 being the BCM2712 SoC, the Pi 5’s chip.
4.3Audio stack and Ollama
PulseAudio, PortAudio and rtkit go in first, then Ollama’s installer. Its closing line is the honest one: “No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.” That is the operating assumption for this whole project, not a warning to work around.
4.4Does the model actually work?
Before wiring anything to audio, check the brain in isolation. Two prompts, two languages, answered locally on the board:
4.5Whisper, in a virtualenv
faster-whisper pulls in CTranslate2 and ONNX Runtime. Notice the wheels come from piwheels — precompiled for aarch64, which is the difference between a two-minute install and an hour of compiling on the Pi.
4.6The Portuguese voice
Piper’s voices are plain .onnx files pulled from Hugging Face. The PT-BR “faber” medium voice is 60 MB — this is the last thing the build needs from the internet.
4.7Audio is where the real work is
This is the step nobody warns you about. Getting models to run is straightforward; getting a USB microphone and a Bluetooth speaker to behave on Linux is where the hours go. Two separate mixers, tuned independently in alsamixer — playback on the PulseAudio device, capture on the USB PnP Sound Device.
Set the capture gain too low and Whisper transcribes silence; too high and the mic clips into noise that the model happily hallucinates words out of. 62 with +14.88 dB was the sweet spot for this microphone in this room — expect to find your own.
4.8Wiring the three together
The script is short because each component does one thing. record_audio() captures at the mic’s native 44.1 kHz and resamples to the 16 kHz Whisper expects; transcribe() hands the array to Whisper; ask_llm() keeps a six-turn history and posts to Ollama’s local REST API.
# the resample line that makes the USB mic usable by Whisper audio = resample_poly(audio.flatten(), 160, 441) # 44100 → 16000 Hz # Whisper loaded once, quantised to int8 for CPU whisper = WhisperModel(WHISPER_MODEL, device="cpu", compute_type="int8")
4.9First run
The moment of truth — python3 assistant/main.py, and Whisper starts loading.
Once it worked by hand, the last step was making it permanent: a systemd unit so the assistant starts with the board and restarts if it ever dies. That is the service you saw booting at the top of this page.
05Software stack
| Stage | Tool | Model |
|---|---|---|
| Speech-to-Text | faster-whisper | large-v3 |
| Language Model | Ollama | qwen2.5:3b |
| Text-to-Speech | Piper TTS | pt_BR-faber-medium |
| Wake Word | openWakeWord | hey_jarvis |
| Audio I/O | sounddevice / PulseAudio | 44.1→16 kHz |
| Runtime | ONNX Runtime | CPUExecutionProvider |
06RAM footprint
Comfortably within the 8GB Pi 5’s budget.
07Roadmap
- Custom wake word trained in Portuguese
- Local RAG over personal documents (ChromaDB)
- Home Assistant integration via local API
- Web interface with Open WebUI
- Vision support (camera + llava) and YOLOv8 smart alerts
08Built with
- Ollama — local LLM runtime
- faster-whisper — optimized Whisper for CPU
- Piper TTS — offline text-to-speech by Nabu Casa
- openWakeWord — open-source wake word detection
