> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getbased.health/llms.txt
> Use this file to discover all available pages before exploring further.

# AI provider internals

> Developer contract for direct provider routing, cloud consent, private transports, Local AI adapters and lifecycle, model storage, OAuth, and web-search boundaries.

AI provider code touches health context, API keys, browser transport, private TEE modes, web search, import parsing, and chat streaming. Treat it as a privacy/security surface, not just a dropdown.

## Provider map

CLI agents are a distinct backend, not another `/chat/completions` endpoint. See [CLI agent provider internals](/developers/cli-agent-providers) for the Companion, structured tools, gateways, and lifecycle authorization.

| Provider | Main path | Notes |
| - | - | - |
| OpenRouter | direct API / OAuth token | PKCE/state validation for OAuth; supports web search where model/provider allows |
| PPQ | direct API key | anonymous account/top-up flow; regular and private model selections are separate |
| PPQ Private TEE | Tinfoil/private endpoint wrapper | browser-encrypted request body, attestation/lock state, web search disabled |
| Routstr | Cashu/Nostr node session | wallet and node balances are separate; real-funds paths need safety tests |
| Venice | standard or Encrypted TEE Mode | Encrypts message content; requires TDX binding, Intel DCAP, NVIDIA NRAS, and NRAS-token signature checks; metadata remains visible; web search/images are disabled |
| Custom API | direct browser request | endpoint must support browser CORS; no getbased relay fallback |
| Local AI | direct browser request to a compatible endpoint | endpoint can be loopback, LAN, remote, or cloud-tagged; Ollama, LM Studio, and Unsloth Studio have identified adapter capabilities |

## Core modules

The shared chat model picker (`chat-model-controls.ts`, `chat-model-preferences.ts`) follows the active provider and privacy mode immediately. Recommended groups are scoped to that catalog, not merged with other saved providers. `reasoning-capabilities.ts` normalizes explicit reasoning metadata and narrow documented fallbacks; unsupported models must not get an invented effort control. The same selection is carried inside attested private transports when supported.

OpenRouter supplies per-model controls, Venice supplies reasoning metadata, and PPQ/Routstr/Custom retain supported compatible metadata. Ollama and LM Studio can expose native thinking controls. Jan, Unsloth Studio, llama.cpp, and other compatible endpoints only get controls their metadata/serving contract supports. Built-in reasoning is not evidence of configurable effort.

For non-chat feature calls use `ai-feature-routing.ts` so CLI selection, global pause, text/image capability, and no-silent-fallback behavior remain consistent. Keep direct transport functions in the API modules; do not implement a second CLI dispatcher per feature.

| Module | Role |
| - | - |
| `api.ts` | provider routing, model lists, streaming calls, PPQ private transport creation |
| `api-provider-storage.ts` | localStorage keys for provider, model, private-mode, and key state |
| `provider-panel-renderers.ts` | Settings → AI panel markup, PPQ Private TEE UI, model dropdowns |
| `provider-model-controls.ts` | dropdown changes, model persistence, E2EE/private-mode switching |
| `provider-ppq-panels.ts` | PPQ account, balance, top-up, and private-model panel behavior |
| `provider-local-ai-controls.ts` | Local AI URL validation, discovery UI, CORS help, Advisor, and server switching |
| `local-ai-provider-registry.ts` | normalized Local AI adapter registry and detection precedence |
| `local-ai-provider-{ollama,lmstudio,openai-compatible}.ts` | backend discovery, inference, context, performance, and unload implementations |
| `local-ai-provider-shared.ts` | URL normalization, location classification, model normalization, and secret redaction |
| `local-ai-discovery.ts` | cached discovery orchestration and runtime metadata updates |
| `api-local.ts` | Local AI prompt sizing, context planning, native request selection, and normalized diagnostics |
| `local-ai-lifecycle.ts` | memory-only runtime tracking and same-machine model handoff/unload |
| `chat-attestation.ts` | lock/attestation status shown in chat |
| `api-custom.ts` | Browser-direct Custom model discovery, validation, and chat transport |
| `cloud-ai-consent.ts` | Versioned recipient/origin-scoped approval before cloud AI or voice transport |
| `proxy-runtime.ts` | Narrow compatibility-service routing for non-AI operations such as operated NRAS/wearables/CAMS |

## Local AI adapter contract

`LOCAL_AI_PROVIDER_ADAPTERS` registers Ollama, LM Studio, Unsloth Studio, and generic compatible adapters with a stable `id`, user-facing `label`, normalized `capabilities`, and `discover()` method. Adapters can also implement `infer()`, `prepareNativeRequest()`, `loadWithContext()`, and `unload()`.

| Capability | Ollama | LM Studio | Unsloth Studio | Generic compatible endpoint |
| - | - | - | - | - |
| Native discovery | Yes | Yes | Compatible models + identified runtime | No |
| Loaded-model state | Yes | Yes | Yes when status is available | No |
| Context override | Yes | Yes | No | No |
| Native streaming | Yes | No | Compatible streaming | No native path |
| Structured output | Yes | No | Compatible request hint | Compatible request hint |
| Native performance stats | Yes | Yes | Endpoint-dependent | Endpoint-dependent |
| Model unload | Yes | Yes | No | No |

Code outside the adapters must branch on capabilities or optional methods, not hard-coded product names. Unknown providers fall back to the compatible adapter.

### Detection order

Discovery normalizes the base URL before caching or probing.

1. Probe LM Studio's native API and `/v1/models` in parallel.
2. If the native LM Studio API succeeds, merge its richer metadata with compatible model IDs and do not send Ollama-only probes.
3. If compatible models identify `owned_by: unsloth-studio`, query `/api/inference/status`, enrich loaded/context/vision state when available, and do not send Ollama-only probes.
4. Otherwise probe Ollama's `/api/tags` and `/api/ps` endpoints.
5. Prefer Ollama when it is available with models; otherwise use the identified or generic compatible result.

Successful discovery is cached for 30 seconds and failed discovery for 3 seconds. Explicit connection tests force refresh. Cache entries include the normalized URL and API key so results cannot cross credentials.

### Location and privacy classification

`getLocalAiExecutionLocation()` reports:

* `local` for `localhost`, `127.0.0.1`, and IPv6 loopback;
* `lan` for `.local`, private IPv4, link-local, and private/link-local IPv6 hosts;
* `remote` for other hosts; and
* `cloud` for cloud-tagged model names, regardless of the URL.

This is UI disclosure and diagnostic metadata, not endpoint attestation. A remote operator can misrepresent what runs behind an endpoint. Plain HTTP outside loopback must show an unencrypted-transport warning unless an underlying VPN or tunnel provides encryption.

Use the exact browser origin in CORS help. Do not recommend wildcard origins. Mixed-content preflight must continue to reject an HTTP LAN/remote endpoint from an HTTPS page while allowing the browser's loopback exception.

### Context planning and inference

`api-local.ts` estimates prompt tokens, adds a safety reserve, and clamps requested output to the loaded context. If the remaining context cannot satisfy the minimum output, it fails with the current and maximum context sizes instead of sending a predictably truncated request.

Imports and meal-photo analysis opt into discovery and native context planning because a full report or structured nutrition result can exceed a default loaded context. Dense lab tables use a conservative three-characters-per-token estimate; routine chat avoids an extra discovery probe before the first token.

`nutrition-ai-settings.ts` keeps an optional meal-specific model route separate from the main chat model while remaining on the active provider, and filters its catalog to image-capable models. Benchmarks can select models from multiple configured providers. Meal requests use recipient consent kind `meal-photo`, structured JSON output, no automatic retry, and their own abort signal. See [Meals and Nutrition internals](/developers/nutrition-internals).

Ollama can pass `num_ctx`. For a current LM Studio server, `prepareNativeRequest()` selects a target context and `loadWithContext()` unloads the previous instance, loads the model through `/api/v1/models/load`, forces discovery, and returns the context LM Studio actually fitted. `api-local.ts` replans output against that verified context, then generates through streaming `/v1/chat/completions`. A missing load route falls back to native non-streaming inference with a 15-minute ceiling. An unload failure may proceed only after forced discovery proves the old instance is gone; otherwise it fails before allocating a duplicate.

Local streaming gets a 15-minute allowance only for the first reader result because prompt prefill can be silent. Every later read keeps the shared 30-second stall guard. Provider finish reasons and native LM Studio token accounting normalize to `truncated`; PDF text and image imports reject that result before JSON review so a partial marker list cannot be confirmed.

All adapters return the common text/usage result plus optional diagnostics. Generic compatible endpoints and Custom API use direct browser requests to `/v1/chat/completions`.

### Runtime handoff and VRAM

`local-ai-lifecycle.ts` keeps only the last base URL, adapter ID, and model in module memory. Before using a different endpoint/model on the same machine, it discovers the previous runtime and unloads its loaded model when the adapter supports that operation.

Do not persist runtime-use state: it becomes stale after refresh and can expose misleading loaded-model claims. If a supported unload operation fails while changing servers in Settings, keep the new selection unsaved and return an actionable manual-unload error. An adapter without unload support can proceed without automatic release. Never attempt to unload cloud-tagged models or models on a different machine.

Model-test runtime capture is documented in [AI model testing internals](/developers/ai-model-testing-internals).

## Model storage rules

Regular and private/encrypted modes may need separate model keys. Do not let a model selected for a public provider path leak into a private-mode request if the private endpoint has a smaller model list.

When switching encryption/private modes:

* clear or reset incompatible sessions;
* restore the last valid model for that mode when possible;
* hide unavailable controls rather than sending unsupported requests;
* update pricing/usage hints after the final model is selected.

## Web search boundary

Web search can add external search results to the prompt. It must be disabled when the active transport promises private/encrypted prompt handling that would be broken by search, including PPQ Private TEE and Venice Encrypted TEE Mode.

User docs should say this plainly, and UI state should match it: no visible Web toggle when search is unavailable.

## Custom API direct-transport boundary

`api-custom.ts` calls `/models` and `/chat/completions` directly from the browser with `credentials: 'omit'`. `callOpenAICompatibleAPI()` receives `{ useProxy: false }`, and connection errors explicitly state that no server retry occurred.

Remote Custom origins require recipient/origin-scoped cloud-processing consent before chat data is sent. Loopback and private-network Custom origins are treated as local for that consent gate, though the configured machine can still read the request. The endpoint must allow the exact app origin with CORS and must satisfy mixed-content rules.

Do not reintroduce an implicit relay fallback. It would change which operator can see prompts and credentials and would require a new implementation, privacy disclosure, consent route, abuse controls, and security review.

## OpenRouter OAuth invariants

OpenRouter OAuth must bind state to the browser session and use PKCE/state validation. Do not accept callback tokens without matching the pending state. OAuth errors should leave existing manually-entered keys untouched unless the user explicitly clears them.

## Verification checklist

Before shipping provider changes:

* run targeted provider tests for the touched provider;
* run `tests/api-provider-contracts.test.ts` for adapter inference and redaction contracts;
* run `tests/test-provider-local-ai-runtime.ts` for discovery, context, location, and lifecycle behavior;
* run chat streaming tests if transport code changed;
* run Settings panel tests if dropdowns/toggles changed;
* verify Local AI connection copy for loopback, LAN, remote, cloud, CORS, and mixed-content cases;
* verify capability-optional fields render safely for a generic compatible endpoint;
* verify switching same-machine Ollama/LM Studio runtimes unloads only when supported;
* verify Local AI imports distinguish first-token prefill from mid-stream stalls and reject truncated marker lists;
* verify web search visibility for public vs private modes;
* verify wrong/missing API key errors are clear and do not erase saved keys;
* verify private/E2EE attestation status is visible and fail-closed;
* verify no raw keys, tokens, prompts, or setup blobs appear in logs or docs.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.