Hi everyone, I wanted to share my journey in finding the ideal local voice assistant setup for my Home Assistant installation. After experimenting with various options, I’ve discovered a few configurations that might be of interest to others looking for a balance between performance and cost.
I started with the Raspberry Pi 5 and N100, but the latency was frustrating—5-10 seconds for a response just didn’t cut it for a seamless experience. I then considered a gaming PC, but the power consumption and need to run it 24/7 felt excessive. Cloud solutions were fast, but they defeated the purpose of self-hosting and raised privacy concerns.
After months of experimenting, I settled on two configurations that offer a great balance:
-
Orange Pi 5 / RK3588: This setup uses the Rockchip RK3588 with 8GB RAM and runs Llama 3.2 3B at ~15–20 tokens/sec. It’s a solid baseline for a usable voice assistant, significantly faster than the Pi 5. The NPU drivers were a bit tricky to configure, but the result is a reliable and responsive experience.
-
NVIDIA Jetson Orin Nano: This premium option uses an NVIDIA Orin Nano with 8GB RAM and an NVMe SSD, achieving ~40+ tokens/sec. The response is nearly instantaneous, comparable to commercial services like Alexa or Google Nest. The use of NVMe drives was crucial, as SD cards introduced unacceptable delays during cold starts.
I’ve been running these setups reliably using the Wyoming protocol and am considering offering pre-built units to the community at cost. If you’re interested, I’d love to hear your feedback—does the budget-friendly Orange Pi appeal to you, or is the instant-response of the Jetson worth the extra cost?
This project has been incredibly rewarding, and I’m excited to share it with others who might be facing similar challenges in optimizing their local voice assistants. Let’s keep the discussion going and explore how we can further enhance our smart home experiences!