A Multimodal AI Assistant Emerges from a Credit Card-Sized Computer

A new demonstration shows Google's Gemma 4, a multimodal language model, running a fully autonomous visual assistant on NVIDIA's compact Jetson Orin Nano. The system listens, sees, and speaks, deciding for itself when a question requires visual context.

The setup is deceptively simple: a USB webcam, microphone, and speaker connected to the small 8GB Jetson board. A user presses a key to speak. The audio is transcribed locally by Parakeet STT and sent to Gemma 4. The model, without any programmed triggers, evaluates whether answering requires seeing. If so, it instructs the script to capture a webcam image. Gemma then analyzes the scene to inform its verbal response, which is delivered through the Kokoro TTS system.

Developer Asier Arranz has published the complete script on GitHub. Getting it operational involves compiling the efficient llama.cpp inference engine natively on the Jetson to enable the crucial vision capabilities. While the process requires careful management of the board's limited memory, the result is a responsive, integrated AI agent.

For those interested in testing the core language model without the vision component, a pre-built Docker container from the Jetson AI Lab offers a faster start. However, the full visual assistant experience necessitates the native build. This project illustrates the advancing frontier of edge AI, packing sophisticated, decision-making models into increasingly modest hardware.

Source: Hugging Face Blog