Umajin now has LLama.cpp and agent.cpp available as a native hardware accelerated DLL for Windows and MacOS.
In this example video we will show completely local Whisper for speech to text, Gemma 4 E2B for LLM reasoning, pocket TTS for voice cloning and text to speech. These are all running locally in memory with a simple Umajin voice interface which supports interuptions.
There is also an agent framework for tool use and to remember and compact the KV Cache for the thread of a conversation including in this demo my name.
Cheers
David