Testing

A modular, production-grade test + hardening suite. It runs entirely against an isolated local stack and a dedicated redline_test database — never staging or production. Each phase has a single-command runner and explicit pass/fail criteria. The quick command list is on the Cheatsheet.

Phase	What	Runner
0	Foundations: isolated env + guard + hermetic stack	npm run stack:test:up
1	Data seeding (Faker, deterministic, fixtures)	npm run seed
2	End-to-end (Playwright, real Keycloak login)	npm run test:e2e
3	Integration (REST vs real Postgres)	npm run test:integration
4	Load testing (Artillery)	npm run test:load
5	Security / pentest + hardening (OWASP)	npm run test:security

Safety model (read this first)

Everything destructive routes through assertTestEnvironment() (tests/_shared/guard.ts). A runner refuses to start unless: NODE_ENV=test (or TEST_ENV=1), and DATABASE_URL names a database ending in _test, and it's a local host (localhost/127.0.0.1/::1/ postgres), and not on the amrkt.ch denylist. The webapp's dev API base (apps/webapp/src/api.ts) defaults to staging; .env.test overrides it to the local stack so the browser suites can never touch staging.

cp .env.test.example .env.test     # isolated, local-only config

Phase 0 — Foundations

A hermetic stack (Postgres + Keycloak + MCP/REST server) on the redline_test database, isolated under the redline-test compose project.

npm run stack:test:up       # build + start, waits for /healthz + Keycloak
npm run stack:test:down     # stop + remove containers + volumes

Files: docker-compose.test.yml, scripts/test-stack.mjs, .env.test.example, tests/_shared/guard.ts.

Phase 1 — Data seeding

A large, realistic, referentially-consistent dataset built with @faker-js/faker and bulk-inserted via Kysely (reusing the mcp-server's own pool, migrator and schema).

npm run seed                 # defaults: 25k listings, 500 dealers, 50 orgs, 2k buyers …
npm run seed -- --listings 50000 --dealers 1000 --seed 7
npm run seed -- --append     # add to existing data instead of resetting
npm run seed:reset           # truncate every app table (guarded)
npm run seed:fixtures        # provision the known login users in Keycloak

Determinism: a fixed --seed reproduces the dataset byte-for-byte. Idempotency: without --append, seed truncates first.

Known fixtures (logins for E2E + security; password Test1234!):

Login	Role	Purpose
dealer-a@market.test	dealer	owns ~12 listings; "user A" in IDOR tests
dealer-b@market.test	dealer	owns ~12 listings; "user B" in IDOR tests
buyer@market.test	buyer	favorites, saved searches, reviews
mod@market.test	dealer + listings:moderate	moderation flows

Files: tests/seed/{seed,reset,factories,fixtures,keycloak,store}.ts.

Phase 2 — End-to-end (Playwright)

Browser E2E across Chromium / Firefox / WebKit, driving the real webapp against the local stack with a real Keycloak UI login (no token injection). Page objects in tests/e2e/pages, specs in tests/e2e/specs; selectors are data-testid attributes.

npm run stack:test:up && npm run seed:fixtures && npm run seed
npx playwright install        # one-time
npm run test:e2e              # all specs, all browsers
npm run test:e2e:smoke        # @smoke subset (fast lane)
npm run test:e2e:ui           # interactive UI mode

The webapp is served by Playwright's webServer on port 4173 in Vite test mode, pointed at the local stack via apps/webapp/.env.test.local. Journeys: browse + filter + NL search + detail (@smoke), login/logout, favorite → saved list, dealer create/edit/delete, and a permission boundary (two dealers' listings are disjoint — no cross-tenant leak).

Files: tests/e2e/{playwright.config.ts,env.ts,fixtures.ts,webapp-env.mjs,pages/*,specs/*}.

Phase 3 — Integration (REST vs real Postgres)

Exercises the real /v1 REST API wired to the Postgres repositories against a disposable Postgres testcontainer locally (or DATABASE_URL in CI).

npm run test:integration
DATABASE_URL=postgres://… npm run test:integration   # reuse an existing DB (CI)

Auth is injected via a test-only header seam (x-test-sub / x-test-scopes) feeding the real createOAuthPrincipalResolver. Coverage: anonymous reads; 401 (unauthenticated write), 403 (missing scope), create → read-back → owner-scoped list, 400 validation, cross-tenant IDOR (404), ETag/304, NL search, favorites, profile upsert, and 429 rate limiting. The fast path (npm test) stays DB-free.

Files: vitest.integration.config.ts, tests/integration/{globalSetup,restApp,rest.int.test}.ts.

Phase 4 — Load testing (Artillery)

Load-tests the read hot path (search, detail, similar, NL search) against the local stack.

npm run stack:test:up && npm run seed
npm run test:load                       # smoke (1 VU)
npm run test:load -- --profile average  # or stress | spike | soak | ci

Profiles smoke | ci | average | stress | spike | soak. The gated profiles enforce p95 + error-rate thresholds that fail the run (Artillery ensure); stress/spike are exploratory. A guard refuses any non-local target unless allowlisted via LOAD_ALLOW_HOSTS.

Files: tests/load/run.mjs.

Phase 5 — Security / pentest + hardening

Hardening: a securityHeaders middleware on the REST API (X-Content-Type-Options, X-Frame-Options, Referrer-Policy, CSP, HSTS, CORP).

App-layer security tests (tests/integration/security.int.test.ts): security headers, broken access control (missing auth → 401, no-subject token → 401, scope escalation → 403) and SQL-injection resilience — all against real Postgres.

npm run test:security   # npm audit (production deps, High+) + the app security tests

Static + dynamic scans (CI):

Scan	Tool	Lane	Gate
Dependencies	npm audit --omit=dev	every push / PR	High+ (production deps)
Secrets	gitleaks (+ history)	every push / PR	any leak (test allowlist)
SAST	Semgrep (OWASP Top Ten + JS/TS)	every push / PR	ERROR severity
Container	Trivy (mcp-server image)	PR / dispatch / nightly	fixable CRITICAL
DAST	OWASP ZAP baseline	dispatch / nightly	FAIL-level alerts

DAST is also runnable locally:

npm run stack:test:up
tests/security/zap-baseline.sh http://localhost:3000/v1/listings

Files: apps/mcp-server/src/rest/router.ts (securityHeaders), tests/integration/security.int.test.ts, tests/security/zap-baseline.sh, .gitleaks.toml.

CI lanes

• Every push / PR (fast): check (unit) · integration · audit · secrets · sast (PRs also run Chromium @smoke E2E).
• On-demand (Run workflow) + nightly (heavy): full cross-browser E2E · Artillery load · Trivy container · OWASP ZAP dast.

Run the full suite anytime via Actions → CI → Run workflow; the nightly cron runs it automatically.