Concepts

A short tour of the ideas that shape how ker-ai behaves, so the configuration in the SDK and API reference makes sense.

Interaction modes

ker-ai supports two ways to move voice between the app and the backend. Pick per session via mode.

client_text (default)

The device does speech-to-text and text-to-speech locally — on web, through the browser's speech APIs — and only text crosses the wire. The backend receives text and returns text. This is the default because it is cheaper and lower latency for many apps, and it keeps audio on the device.

server_audio

The client streams raw microphone audio to the backend over a WebSocket; the backend runs speech-to-text, the model, and text-to-speech, and streams audio back. Use this when you want consistent, high-quality Malayalam speech that does not depend on the device's built-in voices, or when the platform lacks usable local speech APIs.

Start with client_text. Move a session to server_audio when device speech quality or coverage becomes the limiting factor.

Tool calling

A tool is one of your app's functions, described to the model with a name, a purpose, and a typed parameter schema. The model decides — from what the user said — whether and how to call it. ker-ai runs the call through your handler and folds the result back into the conversation, so speech turns into real action in your app instead of a chatbot reply.

This is the difference between an assistant that talks about your app and one that operates it. See the tool-call loop.

Providers and models

Speech-to-text, the language model, and text-to-speech are each pluggable. A session names a provider and a model for each axis, so you can choose the trade-off — cost, latency, quality, or data residency — without changing your app code. The available options are discoverable at runtime via /api/providers/catalog.

Recommendations

ker-ai is not only reactive. Given a snapshot of your app's state — the current screen, a cart value, how long since the last order — it can proactively surface relevant suggestions through the same assistant surface. You push the snapshot; ker-ai decides what, if anything, is worth nudging.

Language

Malayalam (ml-IN) is the language ker-ai is built and proven around — speech recognition, the model's understanding, and the synthesized voice are all tuned for it first. The architecture is not Malayalam-only: the same modes, tools, and providers carry to other languages by changing language and picking appropriate provider models. But Malayalam is the standard the rest is held to.