LLMs under 10 Watts: Raspberry Pi's new $130 AI HAT

Duration: 08:46

Raspberry Pi's back with a new AI HAT.

This time it adds on 8 GB of RAM and the Hailo 10H for $20 over the original.

Raspberry Pi provided the AI HAT+ 2 that I tested in this video. They did not pay for the video nor have any say in the video's contents. See my review sample policy here: https://github.com/geerlingguy/youtube?tab=readme-ov-file#sponsorships

Resources I mentioned in this video:

- Raspberry Pi AI HAT+ 2 ($130): https://www.raspberrypi.com/products/ai-hat-plus-2/

- Hailo 10H Product Page: https://hailo.ai/products/ai-accelerators/hailo-10h-ai-accelerator/

- Turbo Encabulator: https://www.youtube.com/watch?v=Ac7G7xOG2Ag

- llama.cpp on the Pi 5: https://www.jeffgeerling.com/blog/2024/llms-accelerated-egpu-on-raspberry-pi-5/)

- Qwen3 30B A3B model for 16GB Pi 5: https://huggingface.co/byteshape/Qwen3-30B-A3B-Instruct-2507-GGUF/

- Fujitsu Shrink Detection: https://www.youtube.com/watch?v=flD-WfJ4pUg

- Raspberry Pi AI Camera ($70): https://www.raspberrypi.com/products/ai-camera/

- Raspberry Pi AI HAT+ (original - $110): https://www.microcenter.com/product/687346/product?src=raspberrypi

Support me on Patreon: https://www.patreon.com/geerlingguy

Sponsor me on GitHub: https://github.com/sponsors/geerlingguy

Merch: https://www.redshirtjeff.com

2nd Channel: https://www.youtube.com/@GeerlingEngineering

3rd Channel: https://www.youtube.com/@Level2Jeff

Contents:

00:00 - Pi $130 AI HAT+ 2

00:58 - AI Turbo Encabulator

01:48 - LLM tests and NPU vs CPU

02:50 - How useful are tiny models?

04:22 - Running Qwen3 30B on the 16GB Pi 5

05:28 - Machine vision

06:16 - CPU is much worse here

06:52 - Software support

07:31 - Mixed mode not working for me

08:05 - Hard to recommend