AI client — server-side proxy with SSE streaming

Overview

mouaif is the only thing that holds provider API keys. Provider connections are app-level; model IDs are project-level and reference a provider. The mobile UI POSTs to /api/ai/chat, the server combines the selected project model with its provider connection, calls upstream, and forwards events as SSE.

Ten providers ship today:

• OpenAI compatible — any OpenAI-shaped endpoint (OpenAI, Together, Groq, LM Studio, llama.cpp's llama-server, Ollama's /v1, etc.). API key optional: a local server needs none (see local-openai-servers.md).

• Anthropic — Claude Messages API. API key, or OAuth for Claude Pro/Max accounts.

• Google Gemini — Google AI Studio. API key.

• Ollama — local server. No key.

• OpenRouter — one API key, many models (Anthropic, OpenAI, Google, Meta, Mistral, etc.) over an OpenAI-shaped endpoint. API key, or PKCE sign-in (the server stores the issued key).

• GitHub Copilot — reserved; OAuth-only, requires an active Copilot subscription.

• Azure OpenAI — deployment-based OpenAI endpoint. api-key header + api-version query param. API key.

• Mistral — La Plateforme, OpenAI-shaped. API key.

• Groq — fast inference for open models, OpenAI-shaped. API key.

• DeepSeek — OpenAI-shaped API. API key.

  See docs/features/cloud-providers.md for the four OpenAI-shaped providers.

Usage

Behavior

• SSE in, SSE out. The proxy reads the upstream's SSE (or Ollama's NDJSON) and re-emits the same event names with normalized shapes. The browser does not need to know what provider is behind the URL.

• Apikey only in this commit. Models with auth: 'oauth' produce a typed ENOAUTH error. The OAuth commits add the flow; nothing in this commit stores tokens.

• Reserved provider: github-copilot. Listed in ENDPOINTS and providers, gated by reserved: true, so any attempt to call it returns ENOAUTH. The provider's auth flow ships separately.

• Live model catalog. Each ENDPOINTS entry carries a listModels(cred, signal, baseUrl) that returns a normalized [{ id, label, contextWindow? }]. The chat <select> is populated from this list (see Chat UI). baseUrl is the connection's configured base URL, threaded from the app store so the openai-compatible family lists a local llama.cpp / LM Studio server's own models rather than the hosted OpenAI default. Throws ENO_LIST for providers without an adapter, ENO_APIKEY when a provider requires a credential and none is configured, EUNREACHABLE for network failures (Ollama not running, DNS error), EABORTED for the per-call timeout, and EUPSTREAM (with err.status forwarded) for upstream HTTP errors. The HTTP layer maps these to the right status code so the chat UI can show an actionable message — see Live model list.

• Errors are typed. The proxy maps upstream HTTP errors to EUPSTREAM, network failures to ENETWORK, aborts to EABORTED, unknown providers to EUNKNOWN_PROVIDER, missing keys to ENOAPIKEY, OAuth-marked models to ENOAUTH, and bad input to EBADINPUT / EMODEL_NOT_FOUND. The UI branches on code, not on message.

• Connectivity tests time out. /api/ai/test converts its own ten-second abort into ETIMEDOUT; unrelated aborted chat requests remain EABORTED.

• Tool calls use a multi-turn loop. Native and MCP calls emit tool_call / tool_result, append the result to the upstream conversation, and request another model turn. There is no fixed tool-turn limit: the model can continue until it considers the task complete or the user aborts the request. The loop is wired for the OpenAI-shaped providers (openai-compatible, openrouter, github-copilot) and for Anthropic's native tool shape — buildAnthropicRequest converts the specs to tools and the conversation's tool_calls/tool messages to tool_use/tool_result blocks, and the parser reconstructs tool calls from streamed input_json_delta frames.

• Tool feedback is bounded independently from the transcript. The SSE event, SQLite message, trace, and tool card keep the complete structured result. Only the role: "tool" copy sent back upstream is compacted to 64 KiB by default. Oversized text keeps its beginning and end; subagents return only their final text to the parent model; base64 image payloads are omitted from text because they are attached separately as vision input. Historical tool results use the same compaction when a chat is resumed.

• Interrupted turns do not break continuation. When a stored transcript is sent again, only adjacent, complete tool call/result pairs are reconstructed. An orphan call left by an aborted request is omitted, and missing, duplicate, or provider-specific call IDs are replaced consistently on both sides of the pair. Empty persisted assistant segments are also omitted because strict providers reject an assistant message with no visible content. Tool records and OpenAI-shaped tool declarations are omitted for providers whose native tool loop is not implemented (all except the OpenAI-shaped providers and Anthropic). This prevents strict endpoints from returning 400 Bad Request when the user continues the chat or changes models/providers.

• [DONE] sentinel is suppressed. OpenAI uses the literal [DONE] to end a stream; the parser drops it so it does not show up as passthrough in the UI.

• Usage accumulates last-wins within a round, summed across rounds. Each upstream round-trip's usage is tracked per round and folded into the turn aggregate once, when the stream ends. promptTokens is last-report-wins (every round re-sends the full conversation, so the final round's prompt is the real footprint). completionTokens and providerCost sum across tool rounds (each round's output is genuinely new) but are last-wins within one round — so an OpenAI-compatible gateway that stamps a running usage/cost total on every chunk (instead of only the final chunk per stream_options.include_usage) is not double-counted. Anthropic's message_start/message_delta usage frames feed the same per-round trackers; its per-round cost snapshot still rides usage_output deltas.

• Subagent delegated cost surfaces per nested round. Every nested round that finishes emits a usage_update SSE event carrying that round's cost increment ({ cost: { known, total, currency }, source: 'subagent' }), so the chat UI folds it into the running header "Total" pill while the delegated run is still working. A subagent call whose nested run uses tools (several billed upstream rounds) therefore updates several times instead of once at the end. The nested run's completion adds only the cost the round reports did not cover, the parent done event later carries the total inside the final segment's remainder, and the running delta is cleared — nothing is double-counted. See usage-metrics.md.

• Anthropic prompt caching is enabled for API-key and OAuth models. Every request marks the combined system block, the last native tool definition, and the penultimate message with cache_control: { type: 'ephemeral' } (object blocks only, so plain strings are wrapped) using generally available prompt caching; OAuth retains its own required beta header, while API-key requests need no caching beta header. The message breakpoint matters: Anthropic ignores a cache_control breakpoint whose prefix is below the per-model minimum cacheable length (4096 tokens for Sonnet 4 / Opus 4 / Haiku 4.5), and the replayed history is what pushes the prefix over the threshold — so the cache engages from the second request of a conversation even with no tools enabled. Cache read/write token counts from message_start are folded into the per-round snapshots and the final done usage block as cacheReadTokens / cacheCreationTokens, and the cost layer prices them at Anthropic's discounted rates. See prompt-caching.md.

• Project model records are identity metadata only. A .mouaif.json model entry ({ id, provider, label, contextWindow, pricing, thinking, thinkingLevel, maxOutputTokens }) is merged under the app-level provider connection, never over it: transport (baseUrl) and credentials (apiKey, auth, oauthAccount, headers, staticHeaders, authHeader, token, accessToken) always come from the app store. The merge is an allow-list (projectModelRecord() in src/util.js), so a field added later is ignored until it is listed rather than silently flowing through. Every path that hydrates a project model — the chat model, an agent's pinned model, and the approval-card override — goes through it, because the project file is committed with the project and is editable by anything with write access to it.

• POST /api/ai/chat takes an optional providerId pin. A project model is looked up by id, which is not unique across providers; when the caller knows which connection a model came from it also sends providerId, and resolution requires provider === providerId (falling back to the same id-plus-provider rule that already lets a live-catalog model be used without persisting it). The chat itself omits the pin — a chat stores its own providerId/modelId pair — while surface-level callers such as the Inspector's Intent panel send it.

Related

• Settings storage: docs/features/app-and-project-settings.md.