Skip to main content
AI provider code touches health context, API keys, browser transport, private TEE modes, web search, import parsing, and chat streaming. Treat it as a privacy/security surface, not just a dropdown.

​
Provider map

​
Core modules

​
Local AI adapter contract

LOCAL_AI_PROVIDER_ADAPTERS registers three adapters with a stable id, user-facing label, normalized capabilities, and discover() method. Adapters can also implement infer(), prepareNativeRequest(), loadWithContext(), and unload(). Code outside the adapters must branch on capabilities or optional methods, not hard-coded product names. Unknown providers fall back to the compatible adapter.

​
Detection order

Discovery normalizes the base URL before caching or probing.
  1. Probe LM Studio’s native API and /v1/models in parallel.
  2. If the native LM Studio API succeeds, merge its richer metadata with compatible model IDs and do not send Ollama-only probes.
  3. Otherwise probe Ollama’s /api/tags and /api/ps endpoints.
  4. Prefer Ollama when it is available with models; otherwise use the compatible result.
Successful discovery is cached for 30 seconds and failed discovery for 3 seconds. Explicit connection tests force refresh. Cache entries include the normalized URL and API key so results cannot cross credentials.

​
Location and privacy classification

getLocalAiExecutionLocation() reports:
  • local for localhost, 127.0.0.1, and IPv6 loopback;
  • lan for .local, private IPv4, link-local, and private/link-local IPv6 hosts;
  • remote for other hosts; and
  • cloud for cloud-tagged model names, regardless of the URL.
This is UI disclosure and diagnostic metadata, not endpoint attestation. A remote operator can misrepresent what runs behind an endpoint. Plain HTTP outside loopback must show an unencrypted-transport warning unless an underlying VPN or tunnel provides encryption. Use the exact browser origin in CORS help. Do not recommend wildcard origins. Mixed-content preflight must continue to reject an HTTP LAN/remote endpoint from an HTTPS page while allowing the browser’s loopback exception.

​
Context planning and inference

api-local.js estimates prompt tokens, adds a safety reserve, and clamps requested output to the loaded context. If the remaining context cannot satisfy the minimum output, it fails with the current and maximum context sizes instead of sending a predictably truncated request. Imports opt into discovery and native context planning because a full report can exceed a default loaded context. Dense lab tables use a conservative three-characters-per-token estimate; routine chat avoids an extra discovery probe before the first token. Ollama can pass num_ctx. For a current LM Studio server, prepareNativeRequest() selects a target context and loadWithContext() unloads the previous instance, loads the model through /api/v1/models/load, forces discovery, and returns the context LM Studio actually fitted. api-local.js replans output against that verified context, then generates through streaming /v1/chat/completions. A missing load route falls back to native non-streaming inference with a 15-minute ceiling. An unload failure may proceed only after forced discovery proves the old instance is gone; otherwise it fails before allocating a duplicate. Local streaming gets a 15-minute allowance only for the first reader result because prompt prefill can be silent. Every later read keeps the shared 30-second stall guard. Provider finish reasons and native LM Studio token accounting normalize to truncated; PDF text and image imports reject that result before JSON review so a partial marker list cannot be confirmed. All adapters return the common text/usage result plus optional diagnostics. Generic compatible endpoints use direct browser requests to /v1/chat/completions; they do not go through the Custom API proxy.

​
Runtime handoff and VRAM

local-ai-lifecycle.js keeps only the last base URL, adapter ID, and model in module memory. Before using a different endpoint/model on the same machine, it discovers the previous runtime and unloads its loaded model when the adapter supports that operation. Do not persist runtime-use state: it becomes stale after refresh and can expose misleading loaded-model claims. If a supported unload operation fails while changing servers in Settings, keep the new selection unsaved and return an actionable manual-unload error. An adapter without unload support can proceed without automatic release. Never attempt to unload cloud-tagged models or models on a different machine. Model-test runtime capture is documented in AI model testing internals.

​
Model storage rules

Regular and private/encrypted modes may need separate model keys. Do not let a model selected for a public provider path leak into a private-mode request if the private endpoint has a smaller model list. When switching encryption/private modes:
  • clear or reset incompatible sessions;
  • restore the last valid model for that mode when possible;
  • hide unavailable controls rather than sending unsupported requests;
  • update pricing/usage hints after the final model is selected.

​
Web search boundary

Web search can add external search results to the prompt. It must be disabled when the active transport promises private/encrypted prompt handling that would be broken by search, including PPQ Private TEE and Venice Encrypted TEE Mode. User docs should say this plainly, and UI state should match it: no visible Web toggle when search is unavailable.

​
Custom API proxy boundary

Custom API uses api/proxy.js, not a raw browser call to arbitrary URLs. The proxy must keep:
  • method restrictions;
  • CORS restrictions;
  • private-network/public-host checks;
  • target allowlist or explicit validation;
  • no logging of prompt bodies or API keys.
Any relaxation of this boundary needs a security review and tests.

​
OpenRouter OAuth invariants

OpenRouter OAuth must bind state to the browser session and use PKCE/state validation. Do not accept callback tokens without matching the pending state. OAuth errors should leave existing manually-entered keys untouched unless the user explicitly clears them.

​
Verification checklist

Before shipping provider changes:
  • run targeted provider tests for the touched provider;
  • run tests/api-provider-contracts.test.js for adapter inference and redaction contracts;
  • run tests/test-provider-local-ai-runtime.js for discovery, context, location, and lifecycle behavior;
  • run chat streaming tests if transport code changed;
  • run Settings panel tests if dropdowns/toggles changed;
  • verify Local AI connection copy for loopback, LAN, remote, cloud, CORS, and mixed-content cases;
  • verify capability-optional fields render safely for a generic compatible endpoint;
  • verify switching same-machine Ollama/LM Studio runtimes unloads only when supported;
  • verify Local AI imports distinguish first-token prefill from mid-stream stalls and reject truncated marker lists;
  • verify web search visibility for public vs private modes;
  • verify wrong/missing API key errors are clear and do not erase saved keys;
  • verify private/E2EE attestation status is visible and fail-closed;
  • verify no raw keys, tokens, prompts, or setup blobs appear in logs or docs.