Skip to main content
getbased uses two test layers:
  • Vitest wraps the node-side helper fixtures and fast logic checks.
  • Playwright checks the native compatibility entry and assembled static SvelteKit app headlessly. Shipping acceptance covers Chromium, Firefox, desktop WebKit and mobile WebKit; use the project/configuration that owns the behavior.
Routstr/Cashu wallet work has an extra release gate because it touches bearer money. Before changing wallet funding, token export/import, seed restore, node deposit/top-up, refunds, or Cashu TS migration code, read Routstr/Cashu safety. All first-party tests are authored in TypeScript. Some legacy browser fixtures retain classic-script execution semantics; Playwright executes those through tests/playwright/browser-script-runner.ts so they run in the same browser suite as native Playwright specs. Generated marker catalogs have dedicated drift gates:
Use the matching :build command after editing authoring modules, then run the focused identity, placement, persistence, or terminology tests relevant to the diff. GitHub Actions owns the exhaustive browser and combined-coverage run.

Test setup and assertions

For CLI-provider and chat-workspace changes, cover the Companion verification checklist and chat ownership checks. Test discovery-only versus installation-authorized credentials separately, unavailable models/targets, canceled uploads, profile switches before the first token, and source-toggle enforcement in tools. Use mocks for routine CI rather than real health records or paid inference. Native OS service installation and OS file dragging require separate platform checks. The shipping build enforces its offline-cache resource budget; npm run performance:check -- --artifact dist separately measures startup and incremental Settings/Body transfer with synthetic profiles. The native source-mode cold-load budgets remain separate checks. cold-load-budget.spec.ts attaches the per-resource inventory for attribution. Record any deliberate feature cost explicitly instead of removing assertions or silently widening the budget. Exhaustive browser and combined-coverage runs belong in GitHub Actions, not an unapproved local full-matrix run. Install dependencies with Node.js 24 and npm ci. npm test compiles the app and fixtures before Vitest. Before direct Playwright or Node commands after source edits, run npm run typescript:build; keep runtime imports and fixture fetch URLs ending in .js or .mjs. Native unit tests use Vitest assertions. For example:
Legacy Node fixtures can use createLegacyAssertions() from tests/helpers/legacy-assertions.ts through its .js runtime specifier; the helper returns assert and { pass, fail } results. Classic browser fixtures listed in scripts/classic-browser-fixture-entries.json keep their assertions inside a typed runBrowserFixture() wrapper. The build validates the wrapper and emits its body for the browser-script runner, preserving return values and execution mode. Keep every original assertion when adapting a fixture to TypeScript, and include useful failure details. Prefer native Vitest or Playwright for new tests.

Svelte, styling and production acceptance

After changing authored presentation, compile and run npm run styles:check and npm run svelte:check as applicable, plus strict application/test types and relevant behavior tests. Use npm run production:check for a shipping build change. Do not replace frozen native receipts, original assertions or cache/performance/coverage floors merely because the renderer changed. For one static acceptance spec, build once and select that spec explicitly:
npm run test:sveltekit builds first and runs its configured acceptance set; a focused result is not complete release acceptance. The root service-worker protocol fixture at tests/pwa/update-acceptance.spec.ts has its own configuration, playwright.pwa.config.ts. In GitHub Actions, shipping desktop/mobile WebKit UI and real Worker/OPFS acceptance run on macOS Cocoa. The OPFS lane uses persistent profiles and a required write/flush/read control across a browser restart. Chromium/Firefox shipping acceptance and native suites retain Linux placement. Case inventories and the aggregate check reject missing or duplicate receipts. A focused Linux WPE diagnostic does not certify shipping Cocoa behavior. See the app’s testing policy for current partitions and release evidence.

Merge checks and release evidence

PR and main-push checks use an affected-change plan. The merge tier runs selected unit/native regressions, Chromium shipping shards when affected, and the applicable resource budgets. Nightly and manual full runs retain the complete regression/coverage matrix, including shipping Firefox and desktop/mobile WebKit. A green merge check is not a measurement of whole-project coverage or proof that every browser/provider was qualified. Explicit release-evidence.yml dispatch or the run-release-evidence PR label reuses full tests and adds real WASM model checks and isolated container runtime checks. It does not deploy. Installed-agent authentication, paid model access, native service lifecycle, and physical hardware require their own live evidence. Do not describe fixtures as successful live-provider tests. Recent focused contracts include body-measurement.test.ts, body-measurement-cycle.test.ts, marker-baseline.test.ts, the report-feed model/runtime and HTTP/PDF end-to-end specs, backup-client-bundle.test.ts, backup portability, health-tool permission/receipt/recovery tests, and the full-surface Sync compatibility scenarios. Consult the app testing policy for current owners, commands, inventories, budgets, and evidence tiers.

Notable test files

All test files live in the tests/ directory. Native browser specs live under tests/playwright/; static artifact and offline workflow specs live under tests/pwa/; fixture scripts live as tests/test-*.ts. The table calls out representative ownership areas rather than duplicating the complete file listing from the repository. A few tests run node-side (no browser) — pure-helper unit tests + node script guards. Marked node in the table; browser-driven coverage is owned by Playwright specs. The landing page test (test-landing.js) lives in the get-based-site repo.

Local AI and model-testing checks

Run the focused native suites while iterating:
tests/test-provider-local-ai-runtime.ts is a legacy browser fixture executed through the Playwright browser-script runner. While iterating, run its focused wrapper and any directly affected Settings, import, or service-worker specs. GitHub Actions owns the exhaustive browser matrix.

Full CI runner

./run-tests.sh runs repository preflight checks, Vitest, the origin guard, and the full Chromium Playwright suite. Use Node.js 24, installed dependencies, and freshly compiled runtime modules. It starts an isolated server on an available port rather than reusing an arbitrary process on port 8000; reuse requires an explicit REUSE_TEST_SERVER=1. Routine local checks should target the changed behavior. The full runner is guarded outside CI because browser profiles and coverage snapshots can write substantial data. An exceptional local full run requires explicit approval for that write volume and this opt-in:
Pull-request CI selects affected tests and runs the relevant gates. Main-branch and manually requested exhaustive CI runs own the full browser/coverage matrix; affected-only results do not constitute full coverage evidence.

Coverage status

./run-tests.sh and the full Playwright command are intentionally guarded outside CI because they run the complete browser matrix and can produce substantial disk writes. Local development should use change-scoped Vitest files and Playwright specs. Run the guarded full matrix only after explicit approval to set GETBASED_ALLOW_HIGH_WRITE_TESTS=1; routine exhaustive verification belongs to GitHub Actions. With CI authorization, COVERAGE=1 ./run-tests.sh combines browser collection with the explicit native function-coverage receipt. The custom unit provider uses public native V8 collection, preserving exact compiled source mappings, raw captures and test membership. Its coverage-functions.json records function identities and execution counts; it does not infer native statement, branch or byte coverage. The retained production-coverage artifact includes per-file/feature results and revision information. Critical-module coverage is a separate Istanbul gate. See the coverage contract before changing producers or consumers. Running node scripts/playwright-coverage.mjs directly still falls back to the legacy high-surface Chromium sampler when no Playwright suite shards are present. The full COVERAGE=1 ./run-tests.sh path requires suite shards so coverage regressions in the Playwright instrumentation fail clearly. Combined coverage is enforced by scripts/coverage-baseline.json: the current global function floor is 89%, with independent floors for 17 feature groups. COVERAGE_MIN can demand a stricter global floor but cannot lower the committed baseline. Missing feature measurements fail the gate. Runtime coverage uses emitted JavaScript coordinates consistently across Node and Chromium. Svelte components are inventoried once at their native-browser emission; only framework functions positively identified by authoritative source maps are separated. Missing or ambiguous mappings must not improve coverage. Byte coverage is distinct from line/branch coverage, and native function-only evidence must not be presented as those metrics. A separate critical-module Vitest configuration enforces line, statement, function, and branch thresholds for selected safety-sensitive modules; those percentages do not describe the entire app. See vitest.critical.config.ts. Keep gate thresholds intact when adding tests or refactoring.

Accessibility regression scan

tests/test-a11y-axe.ts uses the pinned local axe-core 4.13 dependency and runs axe.run() against the live rendered surfaces. It does not fetch axe from a CDN. Severity policy:
  • critical / serious → test fails on regression vs baseline
  • moderate / minor → logged but doesn’t block (axe leans opinionated at those tiers)

Baseline-locked gate

The gate is baseline-relative, and the current critical/serious baseline is zero. Any critical or serious rule at any scanned stop is therefore a regression. Baseline lives at tests/.a11y-baseline.json:
The test fails when any critical/serious rule’s count exceeds the baseline. New rules with non-zero counts ARE a regression (the suite never saw them before). Improvements (current < baseline) pass and emit a hint to refresh the baseline. _axeVersion must match both the local dependency and PINNED_AXE_VERSION in the fixture. Update and review all three together when bumping axe because rule changes can appear as regressions.

Refreshing the baseline

After a wave of fixes that legitimately drops violation counts:
tests/playwright/a11y-axe-browser.spec.ts pipes the env var into the page context before loading the fixture. The test prints a ▶ {...} JSON line — copy it over the critical/serious/moderate/minor blocks in tests/.a11y-baseline.json. Lower counts only after fixing and verifying the corresponding violations. Raising the baseline would accept regressions and requires explicit review. If the baseline file is missing entirely, the test treats the first run as “establish baseline” and prints the JSON to stdout for paste-back.

Running a single test in the browser

Prefer a focused Playwright spec. For a classic browser fixture, compile first and open an isolated demo/test profile at http://localhost:8000/app, then use the console:
Results appear in the console. This works only for compiled classic fixtures in the explicit manifest; Node fixtures such as test-cycle-improvements.ts belong in the Node/Vitest harness. Do not use Function() to evaluate arbitrary TypeScript or module source.

Writing new tests

When adding a feature or fixing a bug, cover the behavior in the relevant TypeScript test. If none fits, create a focused your-feature.test.ts or Playwright spec. Use source inspection only for contracts that truly depend on source structure; runtime assertions should prove observable behavior. What to cover:
  • Source inspection — verify the function or pattern exists in the source:
  • DOM state — check that elements render correctly:
  • Function behavior — call window-exported functions and check results:
  • CSS rules — verify styles are applied (use getComputedStyle or inspect stylesheets):
  • Persistence — in an isolated fixture, save a known record through the feature’s API, reload the page, and assert that the same reviewed values are restored. Exercise failure paths separately. Checking that a key might exist, or using an always-true fallback, does not verify persistence.

Browser automation and manual checks

Playwright can test synthetic drag-and-drop, file uploads, mocked streaming responses, and IndexedDB persistence across reloads. Keep these tests deterministic with isolated storage and controlled provider responses. A fixture runner resetting between files does not prevent a dedicated spec from testing reload persistence. Manual platform checks remain useful for native OS file dialogs and service installation, real cameras/microphones and hardware, mobile PWA behavior, and live provider compatibility. Paid inference and real wallet funds require deliberate authorization and isolated test data; they are not routine CI requirements.