feat: add MCP server, A2A protocol, auto-combo engine & VS Code extension

Introduce full AI orchestration ecosystem:
- MCP Server with 16 tools, scoped auth, and audit logging
- A2A v0.3 server with JSON-RPC 2.0, SSE streaming, and task manager
- Auto-Combo engine with 6-factor scoring and self-healing
- VS Code extension with smart dispatch and budget tracking
- Harden CI pipeline: add static checks, remove continue-on-error
- Add translator schema validation tests
- Update .gitignore and CHANGELOG for release checklist
This commit is contained in:
diegosouzapw
2026-03-04 18:45:02 -03:00
parent 5ecef5c90c
commit bddec84f4e
26 changed files with 1311 additions and 51 deletions
+6 -2
View File
@@ -22,6 +22,12 @@ jobs:
cache: npm
- run: npm ci
- run: npm run lint
- run: npm run check:cycles
- run: npm run check:route-validation:t06
- run: npm run check:any-budget:t11
- run: npm run check:docs-sync
- run: npm run typecheck:core
- run: npm run typecheck:noimplicit:core
security:
name: Security Audit
@@ -127,7 +133,6 @@ jobs:
cache: npm
- run: npm ci
- run: npm run test:integration
continue-on-error: true
test-security:
name: Security Tests
@@ -144,4 +149,3 @@ jobs:
cache: npm
- run: npm ci
- run: npm run test:security
continue-on-error: true
+2 -1
View File
@@ -63,6 +63,7 @@ docs/*
!docs/TASK_NEBIUS_BACKEND_ENABLEMENT.md
!docs/frontend-backend-provider-gap-report.md
!docs/openapi.yaml
!docs/RELEASE_CHECKLIST.md
!docs/PLANO-IMPLANTACAO.md
!docs/TASKS.md
!docs/FASE-*.md
@@ -106,7 +107,7 @@ app.__qa_backup/
# Production standalone build (created by scripts/prepublish.mjs)
# Conflicts with Next.js App Router detection in dev (root app/ shadows src/app/)
# npm publish still includes it via package.json "files" field
app/
/app/
# Electron (subproject dependency lock and build artifacts)
electron/package-lock.json
+72
View File
@@ -7,6 +7,78 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
---
## [Unreleased]
> ### ✨ Major Feature Release — MCP Server, A2A Protocol, Auto-Combo Engine & VS Code Extension
>
> Full AI orchestration ecosystem: 16 MCP tools, A2A v0.3 server, self-healing Auto-Combo engine, and a VS Code extension with smart dispatch, budget tracking, and human checkpoints.
### 🆕 MCP Server (16 Tools)
- **8 Essential Tools** — `get_health`, `list_combos`, `get_combo_metrics`, `switch_combo`, `check_quota`, `route_request`, `cost_report`, `list_models_catalog`
- **8 Advanced Tools** — `simulate_route`, `set_budget_guard`, `set_resilience_profile`, `test_combo`, `get_provider_metrics`, `best_combo_for_task`, `explain_route`, `get_session_snapshot`
- **Scoped Authorization** — 8 permission scopes (read:health, write:combo, etc.)
- **Audit Logging** — Every tool call logged with duration, arguments, and result
- **IDE Configs** — MCP configuration templates for Antigravity, Cursor, Copilot, Claude Desktop
### 🤖 A2A Server (Agent-to-Agent v0.3)
- **JSON-RPC 2.0** — Full router with `message/send`, `message/stream`, `tasks/get`, `tasks/cancel`
- **Agent Card** — Dynamic `/.well-known/agent.json` with 2 skills
- **Skills** — `smart-routing` (routing explanation, cost envelope, resilience trace, policy verdict) and `quota-management` (natural language quota queries)
- **SSE Streaming** — Real-time task streaming with 15s heartbeat
- **Task Manager** — State machine (submitted→working→completed/failed/canceled), TTL, cleanup
- **Routing Logger** — Decision audit trail with 7-day retention
### ⚡ Auto-Combo Engine
- **6-Factor Scoring** — Quota, health, costInv, latencyInv, taskFit, stability (normalized 0-1)
- **Task Fitness Table** — 30+ models × 6 task types with wildcard boosts
- **4 Mode Packs** — Ship Fast, Cost Saver, Quality First, Offline Friendly
- **Self-Healing** — Progressive cooldown exclusion, probe-based re-admission, incident mode (>50% OPEN)
- **Bandit Exploration** — 5% exploratory routing for discovering better providers
- **Adaptation Persistence** — EMA scoring with disk persistence every 10 decisions
- **REST API** — `POST/GET /api/combos/auto` for CRUD operations
### 🧩 VS Code Extension — Advanced Features
- **MCP Client** — 16 tool wrappers with REST API fallback
- **A2A Client** — Agent discovery, message send/stream, task management
- **Smart Dispatch** — Task type detection, combo recommendation, risk scoring
- **Preflight Dialog** — Risk-based display (auto-skip low, info medium, modal high)
- **Budget Guard** — Session cost tracking with status bar indicator and threshold actions
- **Mode Pack Selector** — Quick-pick UI for switching optimization profiles
- **Health Monitor** — Circuit breaker state change notifications
- **Human Checkpoint** — Multi-factor confidence evaluation with handoff dialog
### 📊 Dashboard Pages
- **MCP Dashboard** — Tool listing, usage stats, audit log with 30s auto-refresh
- **A2A Dashboard** — Agent Card display, skill listing, task history with routing metadata
- **Auto-Combo Dashboard** — Provider score bars, factor breakdown, mode pack selector, incident indicator, exclusion list
### 🔗 Integrations
- **OpenClaw** — Dynamic `provider.order` endpoint at `/api/cli-tools/openclaw/auto-order`
### 🧪 Tests
- **E2E Test Suite** — 6 scenarios (MCP, A2A, Auto-Combo, OpenClaw, Stress 100+50 parallel, Security)
- **Unit Tests** — Essential tools, advanced tools, extension services, Auto-Combo engine, extension advanced features
### 📁 New Files (35+)
| Directory | Files |
| :------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `open-sse/mcp-server/` | `server.ts`, `transport.ts`, `auth.ts`, `audit.ts`, `tools/advancedTools.ts` |
| `src/lib/a2a/` | `taskManager.ts`, `streaming.ts`, `routingLogger.ts`, `skills/smartRouting.ts`, `skills/quotaManagement.ts` |
| `open-sse/services/autoCombo/` | `scoring.ts`, `taskFitness.ts`, `engine.ts`, `selfHealing.ts`, `modePacks.ts`, `persistence.ts`, `index.ts` |
| `vscode-extension/src/services/` | `mcpClient.ts`, `a2aClient.ts`, `policyEngine.ts`, `preflightDialog.ts`, `budgetGuard.ts`, `healthMonitor.ts`, `modePackSelector.ts`, `humanCheckpoint.ts` |
| `src/app/(dashboard)/` | `dashboard/mcp/page.tsx`, `dashboard/a2a/page.tsx`, `dashboard/auto-combo/page.tsx` |
| `docs/` | `mcp-server.md`, `a2a-server.md`, `auto-combo.md`, `vscode-extension.md`, `integrations/ide-configs.md` |
---
## [1.8.1] — 2026-03-03
### 🐛 Bug Fixes
+31 -29
View File
@@ -2,7 +2,7 @@
🌐 **Languages:** 🇺🇸 [English](ARCHITECTURE.md) | 🇧🇷 [Português (Brasil)](i18n/pt-BR/ARCHITECTURE.md) | 🇪🇸 [Español](i18n/es/ARCHITECTURE.md) | 🇫🇷 [Français](i18n/fr/ARCHITECTURE.md) | 🇮🇹 [Italiano](i18n/it/ARCHITECTURE.md) | 🇷🇺 [Русский](i18n/ru/ARCHITECTURE.md) | 🇨🇳 [中文 (简体)](i18n/zh-CN/ARCHITECTURE.md) | 🇩🇪 [Deutsch](i18n/de/ARCHITECTURE.md) | 🇮🇳 [हिन्दी](i18n/in/ARCHITECTURE.md) | 🇹🇭 [ไทย](i18n/th/ARCHITECTURE.md) | 🇺🇦 [Українська](i18n/uk-UA/ARCHITECTURE.md) | 🇸🇦 [العربية](i18n/ar/ARCHITECTURE.md) | 🇯🇵 [日本語](i18n/ja/ARCHITECTURE.md) | 🇻🇳 [Tiếng Việt](i18n/vi/ARCHITECTURE.md) | 🇧🇬 [Български](i18n/bg/ARCHITECTURE.md) | 🇩🇰 [Dansk](i18n/da/ARCHITECTURE.md) | 🇫🇮 [Suomi](i18n/fi/ARCHITECTURE.md) | 🇮🇱 [עברית](i18n/he/ARCHITECTURE.md) | 🇭🇺 [Magyar](i18n/hu/ARCHITECTURE.md) | 🇮🇩 [Bahasa Indonesia](i18n/id/ARCHITECTURE.md) | 🇰🇷 [한국어](i18n/ko/ARCHITECTURE.md) | 🇲🇾 [Bahasa Melayu](i18n/ms/ARCHITECTURE.md) | 🇳🇱 [Nederlands](i18n/nl/ARCHITECTURE.md) | 🇳🇴 [Norsk](i18n/no/ARCHITECTURE.md) | 🇵🇹 [Português (Portugal)](i18n/pt/ARCHITECTURE.md) | 🇷🇴 [Română](i18n/ro/ARCHITECTURE.md) | 🇵🇱 [Polski](i18n/pl/ARCHITECTURE.md) | 🇸🇰 [Slovenčina](i18n/sk/ARCHITECTURE.md) | 🇸🇪 [Svenska](i18n/sv/ARCHITECTURE.md) | 🇵🇭 [Filipino](i18n/phi/ARCHITECTURE.md)
_Last updated: 2026-02-18_
_Last updated: 2026-03-04_
## Executive Summary
@@ -81,8 +81,8 @@ flowchart LR
API[V1 Compatibility API\n/v1/*]
DASH[Dashboard + Management API\n/api/*]
CORE[SSE + Translation Core\nopen-sse + src/sse]
DB[(db.json)]
UDB[(usage.json + log.txt)]
DB[(storage.sqlite)]
UDB[(usage tables + log artifacts)]
end
subgraph Upstreams[Upstream Providers]
@@ -144,7 +144,7 @@ Management domains:
- Providers/connections: `src/app/api/providers*`
- Provider nodes: `src/app/api/provider-nodes*`
- Custom models: `src/app/api/provider-models` (GET/POST/DELETE)
- Model catalog: `src/app/api/models/catalog` (GET)
- Model catalog: `src/app/api/models/route.ts` (GET)
- Proxy config: `src/app/api/settings/proxy` (GET/PUT/DELETE) + `src/app/api/settings/proxy/test` (POST)
- OAuth: `src/app/api/oauth/*`
- Keys/aliases/combos/pricing: `src/app/api/keys*`, `src/app/api/models/alias`, `src/app/api/combos*`, `src/app/api/pricing`
@@ -225,18 +225,19 @@ OAuth provider modules (12 individual files under `src/lib/oauth/providers/`):
## 3) Persistence Layer
Primary state DB:
Primary state DB (SQLite):
- `src/lib/localDb.ts`
- file: `${DATA_DIR}/db.json` (or `$XDG_CONFIG_HOME/omniroute/db.json` when set, else `~/.omniroute/db.json`)
- entities: providerConnections, providerNodes, modelAliases, combos, apiKeys, settings, pricing, **customModels**, **proxyConfig**, **ipFilter**, **thinkingBudget**, **systemPrompt**
- Core infra: `src/lib/db/core.ts` (better-sqlite3, migrations, WAL)
- Re-export facade: `src/lib/localDb.ts` (thin compatibility layer for callers)
- file: `${DATA_DIR}/storage.sqlite` (or `$XDG_CONFIG_HOME/omniroute/storage.sqlite` when set, else `~/.omniroute/storage.sqlite`)
- entities (tables + KV namespaces): providerConnections, providerNodes, modelAliases, combos, apiKeys, settings, pricing, **customModels**, **proxyConfig**, **ipFilter**, **thinkingBudget**, **systemPrompt**
Usage DB:
Usage persistence:
- `src/lib/usageDb.ts`
- files: `${DATA_DIR}/usage.json`, `${DATA_DIR}/log.txt`, `${DATA_DIR}/call_logs/`
- follows same base directory policy as `localDb` (`DATA_DIR`, then `XDG_CONFIG_HOME/omniroute` when set)
- decomposed into focused sub-modules: `migrations.ts`, `usageHistory.ts`, `costCalculator.ts`, `usageStats.ts`, `callLogs.ts`
- facade: `src/lib/usageDb.ts` (decomposed modules in `src/lib/usage/*`)
- SQLite tables in `storage.sqlite`: `usage_history`, `call_logs`, `proxy_logs`
- optional file artifacts remain for compatibility/debug (`${DATA_DIR}/log.txt`, `${DATA_DIR}/call_logs/`, `<repo>/logs/...`)
- legacy JSON files are migrated to SQLite by startup migrations when present
Domain State DB (SQLite):
@@ -505,9 +506,9 @@ erDiagram
Physical storage files:
- main state: `${DATA_DIR}/db.json` (or `$XDG_CONFIG_HOME/omniroute/db.json` when set, else `~/.omniroute/db.json`)
- usage stats: `${DATA_DIR}/usage.json`
- request log lines: `${DATA_DIR}/log.txt`
- primary runtime DB: `${DATA_DIR}/storage.sqlite`
- request log lines: `${DATA_DIR}/log.txt` (compat/debug artifact)
- structured call payload archives: `${DATA_DIR}/call_logs/`
- optional translator/request debug sessions: `<repo>/logs/...`
## Deployment Topology
@@ -522,8 +523,8 @@ flowchart LR
subgraph ContainerOrProcess[OmniRoute Runtime]
Next[Next.js Server\nPORT=20128]
Core[SSE Core + Executors]
MainDB[(db.json)]
UsageDB[(usage.json/log.txt)]
MainDB[(storage.sqlite)]
UsageDB[(usage tables + log artifacts)]
end
subgraph External[External Services]
@@ -550,7 +551,7 @@ flowchart LR
- `src/app/api/providers*`: provider CRUD, validation, testing
- `src/app/api/provider-nodes*`: custom compatible node management
- `src/app/api/provider-models`: custom model management (CRUD)
- `src/app/api/models/catalog`: full model catalog API (all types grouped by provider)
- `src/app/api/models/route.ts`: model catalog API (aliases + custom models)
- `src/app/api/oauth/*`: OAuth/device-code flows
- `src/app/api/keys*`: local API key lifecycle
- `src/app/api/models/alias`: alias management
@@ -582,8 +583,9 @@ flowchart LR
### Persistence
- `src/lib/localDb.ts`: persistent config/state
- `src/lib/usageDb.ts`: usage history and rolling request logs
- `src/lib/db/*`: persistent config/state and domain persistence on SQLite
- `src/lib/localDb.ts`: compatibility re-export for DB modules
- `src/lib/usageDb.ts`: usage history/call logs facade on top of SQLite tables
## Provider Executor Coverage (Strategy Pattern)
@@ -724,23 +726,23 @@ Files are written to `<repo>/logs/<session>/` for each request session.
## 5) Data Integrity
- DB shape migration/repair for missing keys
- corrupt JSON reset safeguards for localDb and usageDb
- SQLite schema migrations and auto-upgrade hooks at startup
- legacy JSON → SQLite migration compatibility path
## Observability and Operational Signals
Runtime visibility sources:
- console logs from `src/sse/utils/logger.ts`
- per-request usage aggregates in `usage.json`
- textual request status log in `log.txt`
- per-request usage aggregates in SQLite (`usage_history`, `call_logs`, `proxy_logs`)
- textual request status log in `log.txt` (optional/compat)
- optional deep request/translation logs under `logs/` when `ENABLE_REQUEST_LOGS=true`
- dashboard usage endpoints (`/api/usage/*`) for UI consumption
## Security-Sensitive Boundaries
- JWT secret (`JWT_SECRET`) secures dashboard session cookie verification/signing
- Initial password fallback (`INITIAL_PASSWORD`, default `123456`) must be overridden in real deployments
- Initial password bootstrap (`INITIAL_PASSWORD`) should be explicitly configured for first-run provisioning
- API key HMAC secret (`API_KEY_SECRET`) secures generated local API key format
- Provider secrets (API keys/tokens) are persisted in local DB and should be protected at filesystem level
- Cloud sync endpoints rely on API key auth + machine id semantics
@@ -762,13 +764,13 @@ Environment variables actively used by code:
## Known Architectural Notes
1. `usageDb` and `localDb` now share the same base directory policy (`DATA_DIR` -> `XDG_CONFIG_HOME/omniroute` -> `~/.omniroute`) with legacy file migration.
2. `/api/v1/route.ts` returns a static model list and is not the main models source used by `/v1/models`.
1. `usageDb` and `localDb` share the same base directory policy (`DATA_DIR` -> `XDG_CONFIG_HOME/omniroute` -> `~/.omniroute`) with legacy file migration.
2. `/api/v1/route.ts` delegates to the same unified catalog builder used by `/api/v1/models` (`src/app/api/v1/models/catalog.ts`) to avoid semantic drift.
3. Request logger writes full headers/body when enabled; treat log directory as sensitive.
4. Cloud behavior depends on correct `NEXT_PUBLIC_BASE_URL` and cloud endpoint reachability.
5. The `open-sse/` directory is published as the `@omniroute/open-sse` **npm workspace package**. Source code imports it via `@omniroute/open-sse/...` (resolved by Next.js `transpilePackages`). File paths in this document still use the directory name `open-sse/` for consistency.
6. Charts in the dashboard use **Recharts** (SVG-based) for accessible, interactive analytics visualizations (model usage bar charts, provider breakdown tables with success rates).
7. E2E tests use **Playwright** (`tests/e2e/`), run via `npm run test:e2e`. Unit tests use **Node.js test runner** (`tests/unit/`), run via `npm run test:plan3`. Source code under `src/` is **TypeScript** (`.ts`/`.tsx`); the `open-sse/` workspace remains JavaScript (`.js`).
7. E2E tests use **Playwright** (`tests/e2e/`), run via `npm run test:e2e`. Unit tests use **Node.js test runner** (`tests/unit/`), run via `npm run test:unit`. Source code under `src/` is **TypeScript** (`.ts`/`.tsx`); the `open-sse/` workspace remains JavaScript (`.js`).
8. Settings page is organized into 5 tabs: Security, Routing (6 global strategies: fill-first, round-robin, p2c, random, least-used, cost-optimized), Resilience (editable rate limits, circuit breaker, policies), AI (thinking budget, system prompt, prompt cache), Advanced (proxy).
## Operational Verification Checklist
+33
View File
@@ -0,0 +1,33 @@
# Release Checklist
Use this checklist before tagging or publishing a new OmniRoute release.
## Version and Changelog
1. Bump `package.json` version (`x.y.z`) in the release branch.
2. Move release notes from `## [Unreleased]` in `CHANGELOG.md` to a dated section:
- `## [x.y.z] — YYYY-MM-DD`
3. Keep `## [Unreleased]` as the first changelog section for upcoming work.
4. Ensure the latest semver section in `CHANGELOG.md` equals `package.json` version.
## API Docs
1. Update `docs/openapi.yaml`:
- `info.version` must equal `package.json` version.
2. Validate endpoint examples if API contracts changed.
## Runtime Docs
1. Review `docs/ARCHITECTURE.md` for storage/runtime drift.
2. Review `docs/TROUBLESHOOTING.md` for env var and operational drift.
3. Update localized docs if source docs changed significantly.
## Automated Check
Run the sync guard locally before opening PR:
```bash
npm run check:docs-sync
```
CI also runs this check in `.github/workflows/ci.yml` (lint job).
+3 -3
View File
@@ -10,7 +10,7 @@ Common problems and solutions for OmniRoute.
| Problem | Solution |
| ----------------------------- | ------------------------------------------------------------------ |
| First login not working | Check `INITIAL_PASSWORD` in `.env` (default: `123456`) |
| First login not working | Set `INITIAL_PASSWORD` in `.env` (no hardcoded default) |
| Dashboard opens on wrong port | Set `PORT=20128` and `NEXT_PUBLIC_BASE_URL=http://localhost:20128` |
| No request logs under `logs/` | Set `ENABLE_REQUEST_LOGS=true` |
| EACCES: permission denied | Set `DATA_DIR=/path/to/writable/dir` to override `~/.omniroute` |
@@ -120,8 +120,8 @@ curl http://localhost:20128/api/monitoring/health
### Runtime Storage
- Main state: `${DATA_DIR}/db.json` (providers, combos, aliases, keys, settings)
- Usage: `${DATA_DIR}/usage.json`, `${DATA_DIR}/log.txt`, `${DATA_DIR}/call_logs/`
- Main state: `${DATA_DIR}/storage.sqlite` (providers, combos, aliases, keys, settings)
- Usage: SQLite tables in `storage.sqlite` (`usage_history`, `call_logs`, `proxy_logs`) + optional `${DATA_DIR}/log.txt` and `${DATA_DIR}/call_logs/`
- Request logs: `<repo>/logs/...` (when `ENABLE_REQUEST_LOGS=true`)
---
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.1.0
info:
title: OmniRoute API
version: 0.5.0
version: 1.8.1
description: |
OmniRoute is a local-first AI API proxy router. It provides an OpenAI-compatible
endpoint that routes requests to multiple AI providers with load balancing,
+9 -2
View File
@@ -58,8 +58,15 @@
"test:plan3": "node --test tests/unit/plan3-p0.test.mjs",
"test:fixes": "node --test tests/unit/fixes-p1.test.mjs",
"test:security": "node --test tests/unit/security-fase01.test.mjs",
"test:e2e": "npx playwright test tests/e2e/*.spec.ts",
"test:vitest": "vitest run open-sse/mcp-server/__tests__/*.test.ts open-sse/services/autoCombo/__tests__/*.test.ts vscode-extension/src/__tests__/*.test.ts",
"check:cycles": "node scripts/check-cycles.mjs",
"check:route-validation:t06": "node scripts/check-route-validation.mjs",
"check:any-budget:t11": "node scripts/check-t11-any-budget.mjs",
"check:docs-sync": "node scripts/check-docs-sync.mjs",
"typecheck:core": "tsc --pretty false -p tsconfig.typecheck-core.json",
"typecheck:noimplicit:core": "tsc --pretty false -p tsconfig.typecheck-noimplicit-core.json",
"test:integration": "node --import tsx/esm --test tests/integration/*.test.mjs",
"test:e2e": "node scripts/run-playwright-tests.mjs test tests/e2e/*.spec.ts",
"test:vitest": "vitest run open-sse/mcp-server/__tests__/*.test.ts open-sse/services/autoCombo/__tests__/*.test.ts",
"test:ecosystem": "node scripts/run-ecosystem-tests.mjs",
"test:coverage": "npx c8 --exclude=open-sse --check-coverage --lines 50 --functions 50 --branches 50 node --import tsx/esm --test tests/unit/*.test.mjs",
"test:all": "npm run test:unit && npm run test:vitest && npm run test:ecosystem && npm run test:e2e",
+177
View File
@@ -0,0 +1,177 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
const cwd = process.cwd();
const defaultRoots = ["src/shared/components", "src/lib/db", "open-sse/translator"];
const roots = process.argv.slice(2).length > 0 ? process.argv.slice(2) : defaultRoots;
const sourceExtensions = [".ts", ".tsx", ".js", ".mjs", ".jsx", ".mts", ".cts"];
function toPosix(filePath) {
return filePath.split(path.sep).join("/");
}
function listSourceFiles(rootDir) {
const absRoot = path.resolve(cwd, rootDir);
if (!fs.existsSync(absRoot)) {
return [];
}
const stack = [absRoot];
const files = [];
while (stack.length > 0) {
const current = stack.pop();
const entries = fs.readdirSync(current, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(current, entry.name);
if (entry.isDirectory()) {
stack.push(fullPath);
continue;
}
if (sourceExtensions.includes(path.extname(entry.name))) {
files.push(path.resolve(fullPath));
}
}
}
return files;
}
function resolveRelativeImport(fromFile, specifier) {
const base = path.resolve(path.dirname(fromFile), specifier);
const ext = path.extname(base);
if (ext && fs.existsSync(base) && fs.statSync(base).isFile()) {
return path.resolve(base);
}
for (const extension of sourceExtensions) {
const candidate = `${base}${extension}`;
if (fs.existsSync(candidate) && fs.statSync(candidate).isFile()) {
return path.resolve(candidate);
}
}
for (const extension of sourceExtensions) {
const candidate = path.join(base, `index${extension}`);
if (fs.existsSync(candidate) && fs.statSync(candidate).isFile()) {
return path.resolve(candidate);
}
}
return null;
}
function extractImportSpecifiers(fileContents) {
const specs = [];
const regex = /\b(?:import|export)\s+(?:[^"'`]*?\sfrom\s*)?["'`]([^"'`]+)["'`]/g;
let match = regex.exec(fileContents);
while (match) {
specs.push(match[1]);
match = regex.exec(fileContents);
}
return specs;
}
function buildGraph(files) {
const fileSet = new Set(files);
const graph = new Map();
for (const filePath of files) {
const code = fs.readFileSync(filePath, "utf8");
const dependencies = new Set();
const importSpecifiers = extractImportSpecifiers(code);
for (const specifier of importSpecifiers) {
if (!specifier.startsWith(".")) continue;
const resolved = resolveRelativeImport(filePath, specifier);
if (!resolved) continue;
if (!fileSet.has(resolved)) continue;
dependencies.add(resolved);
}
graph.set(filePath, dependencies);
}
return graph;
}
function stronglyConnectedComponents(graph) {
const indexMap = new Map();
const lowLinkMap = new Map();
const onStack = new Set();
const stack = [];
const components = [];
let indexCounter = 0;
function strongConnect(node) {
indexMap.set(node, indexCounter);
lowLinkMap.set(node, indexCounter);
indexCounter += 1;
stack.push(node);
onStack.add(node);
for (const neighbor of graph.get(node) || []) {
if (!indexMap.has(neighbor)) {
strongConnect(neighbor);
lowLinkMap.set(node, Math.min(lowLinkMap.get(node), lowLinkMap.get(neighbor)));
} else if (onStack.has(neighbor)) {
lowLinkMap.set(node, Math.min(lowLinkMap.get(node), indexMap.get(neighbor)));
}
}
if (lowLinkMap.get(node) === indexMap.get(node)) {
const component = [];
while (stack.length > 0) {
const candidate = stack.pop();
onStack.delete(candidate);
component.push(candidate);
if (candidate === node) break;
}
components.push(component);
}
}
for (const node of graph.keys()) {
if (!indexMap.has(node)) {
strongConnect(node);
}
}
return components;
}
function isSelfCycle(component, graph) {
if (component.length !== 1) return false;
const [file] = component;
return (graph.get(file) || new Set()).has(file);
}
const files = roots.flatMap((root) => listSourceFiles(root));
const graph = buildGraph(files);
const components = stronglyConnectedComponents(graph);
const cycles = components.filter(
(component) => component.length > 1 || isSelfCycle(component, graph)
);
if (cycles.length === 0) {
console.log(
`[cycles] OK - no cycles detected across ${graph.size} files in: ${roots.join(", ")}`
);
process.exit(0);
}
console.error(`[cycles] FAIL - detected ${cycles.length} strongly connected component(s):`);
for (const component of cycles) {
const sorted = [...component].sort((a, b) => a.localeCompare(b));
console.error(`\n- SCC (${sorted.length} files)`);
for (const filePath of sorted) {
console.error(` - ${toPosix(path.relative(cwd, filePath))}`);
}
}
process.exit(1);
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
const cwd = process.cwd();
const packageJsonPath = path.resolve(cwd, "package.json");
const openApiPath = path.resolve(cwd, "docs/openapi.yaml");
const changelogPath = path.resolve(cwd, "CHANGELOG.md");
function readText(filePath) {
if (!fs.existsSync(filePath)) {
throw new Error(`File not found: ${path.relative(cwd, filePath)}`);
}
return fs.readFileSync(filePath, "utf8");
}
function extractOpenApiVersion(content) {
const lines = content.split(/\r?\n/);
let inInfoBlock = false;
for (const line of lines) {
const trimmed = line.trim();
if (!inInfoBlock) {
if (trimmed === "info:") {
inInfoBlock = true;
}
continue;
}
if (line.length > 0 && !line.startsWith(" ")) {
break;
}
const match = line.match(/^\s{2}version:\s*["']?([^"'\s]+)["']?\s*$/);
if (match) {
return match[1];
}
}
return null;
}
function extractChangelogSections(content) {
const headings = [...content.matchAll(/^##\s+\[([^\]]+)\](?:\s+—\s+.*)?$/gm)];
return headings.map((match) => match[1]);
}
function isSemver(value) {
return /^\d+\.\d+\.\d+$/.test(value);
}
let hasFailure = false;
function fail(message) {
hasFailure = true;
console.error(`[docs-sync] FAIL - ${message}`);
}
try {
const packageJson = JSON.parse(readText(packageJsonPath));
const packageVersion = packageJson.version;
if (!isSemver(packageVersion)) {
fail(`package.json version is not valid semver: "${packageVersion}"`);
} else {
console.log(`[docs-sync] package.json version: ${packageVersion}`);
}
const openApiVersion = extractOpenApiVersion(readText(openApiPath));
if (!openApiVersion) {
fail("could not extract docs/openapi.yaml info.version");
} else if (openApiVersion !== packageVersion) {
fail(`OpenAPI version (${openApiVersion}) differs from package.json (${packageVersion})`);
} else {
console.log(`[docs-sync] openapi.yaml info.version matches: ${openApiVersion}`);
}
const changelogSections = extractChangelogSections(readText(changelogPath));
if (changelogSections.length === 0) {
fail("CHANGELOG.md has no version sections");
} else {
if (changelogSections[0] !== "Unreleased") {
fail('CHANGELOG.md first section must be "## [Unreleased]"');
} else {
console.log("[docs-sync] changelog has top Unreleased section");
}
const semverSections = changelogSections.filter((section) => isSemver(section));
if (semverSections.length === 0) {
fail("CHANGELOG.md has no semver release section");
} else if (semverSections[0] !== packageVersion) {
fail(
`Latest changelog release (${semverSections[0]}) differs from package.json (${packageVersion})`
);
} else {
console.log(
`[docs-sync] latest changelog release matches package version: ${packageVersion}`
);
}
}
} catch (error) {
fail(error instanceof Error ? error.message : String(error));
}
if (hasFailure) {
process.exit(1);
}
console.log("[docs-sync] PASS - documentation version sync is consistent.");
+61
View File
@@ -0,0 +1,61 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
const ROOT = process.cwd();
const API_ROOT = path.join(ROOT, "src", "app", "api");
const FILE_NAME = "route.ts";
const REQUEST_JSON_REGEX = /request\.json\s*\(/;
const VALIDATE_BODY_REGEX = /\bvalidateBody\s*\(/;
/**
* Walk directory recursively and collect route files.
* @param {string} dir
* @returns {string[]}
*/
function collectRouteFiles(dir) {
const entries = fs.readdirSync(dir, { withFileTypes: true });
const files = [];
for (const entry of entries) {
const fullPath = path.join(dir, entry.name);
if (entry.isDirectory()) {
files.push(...collectRouteFiles(fullPath));
continue;
}
if (entry.isFile() && entry.name === FILE_NAME) {
files.push(fullPath);
}
}
return files;
}
if (!fs.existsSync(API_ROOT)) {
console.error(`[t06:route-validation] FAIL - API root not found: ${API_ROOT}`);
process.exit(1);
}
const routeFiles = collectRouteFiles(API_ROOT).sort();
const missingValidation = [];
for (const fullPath of routeFiles) {
const source = fs.readFileSync(fullPath, "utf8");
if (!REQUEST_JSON_REGEX.test(source)) continue;
if (!VALIDATE_BODY_REGEX.test(source)) {
missingValidation.push(path.relative(ROOT, fullPath));
}
}
if (missingValidation.length > 0) {
console.error("[t06:route-validation] FAIL - routes with request.json() without validateBody():");
for (const file of missingValidation) {
console.error(` - ${file}`);
}
process.exit(1);
}
console.log(
`[t06:route-validation] PASS - ${routeFiles.length} route files scanned, all request.json() usages are validated.`
);
+56
View File
@@ -0,0 +1,56 @@
#!/usr/bin/env node
import fs from "node:fs";
import path from "node:path";
const cwd = process.cwd();
/**
* T11 Phase-A budget:
* keep explicit `any` at zero in files already hardened.
*/
const budget = [
{ file: "src/app/api/settings/proxy/route.ts", maxAny: 0 },
{ file: "src/app/api/settings/proxy/test/route.ts", maxAny: 0 },
{ file: "src/shared/components/OAuthModal.tsx", maxAny: 0 },
{ file: "open-sse/translator/index.ts", maxAny: 0 },
{ file: "open-sse/translator/registry.ts", maxAny: 0 },
// Freeze legacy hot spots to avoid any-regression while strict migration continues.
{ file: "src/lib/db/apiKeys.ts", maxAny: 0 },
{ file: "src/lib/db/providers.ts", maxAny: 0 },
{ file: "src/lib/db/settings.ts", maxAny: 0 },
{ file: "open-sse/config/providerRegistry.ts", maxAny: 0 },
{ file: "open-sse/config/providerModels.ts", maxAny: 0 },
{ file: "open-sse/mcp-server/server.ts", maxAny: 0 },
];
const anyRegex = /\bany\b/g;
let hasFailure = false;
for (const item of budget) {
const absolutePath = path.resolve(cwd, item.file);
if (!fs.existsSync(absolutePath)) {
console.error(`[t11:any-budget] FAIL - file not found: ${item.file}`);
hasFailure = true;
continue;
}
const content = fs.readFileSync(absolutePath, "utf8");
const matches = content.match(anyRegex);
const count = matches ? matches.length : 0;
const status = count <= item.maxAny ? "OK" : "FAIL";
if (status === "FAIL") {
hasFailure = true;
}
console.log(
`[t11:any-budget] ${status} - ${item.file} (explicit any: ${count}, budget: ${item.maxAny})`
);
}
if (hasFailure) {
process.exit(1);
}
console.log("[t11:any-budget] PASS - explicit any budget respected.");
+4 -2
View File
@@ -2,6 +2,7 @@
import { spawn } from "node:child_process";
import { setTimeout as delay } from "node:timers/promises";
import { sanitizeColorEnv } from "./runtime-env.mjs";
const port = process.env.DASHBOARD_PORT || process.env.PORT || "20128";
const baseUrl = process.env.OMNIROUTE_BASE_URL || `http://localhost:${port}`;
@@ -33,11 +34,12 @@ async function waitForServerReady() {
async function main() {
let serverProcess = null;
let startedHere = false;
const testEnv = sanitizeColorEnv(process.env);
if (!(await isServerReady())) {
serverProcess = spawn(process.execPath, ["scripts/run-next-playwright.mjs", "dev"], {
stdio: "inherit",
env: process.env,
env: testEnv,
});
startedHere = true;
await waitForServerReady();
@@ -48,7 +50,7 @@ async function main() {
["./node_modules/vitest/vitest.mjs", "run", "tests/e2e/ecosystem.test.ts"],
{
stdio: "inherit",
env: process.env,
env: testEnv,
}
);
+7 -1
View File
@@ -4,6 +4,7 @@ import { existsSync, renameSync } from "node:fs";
import { join } from "node:path";
import {
resolveRuntimePorts,
sanitizeColorEnv,
spawnWithForwardedSignals,
withRuntimePortEnv,
} from "./runtime-env.mjs";
@@ -54,6 +55,11 @@ process.on("uncaughtException", (error) => {
prepareAppDir();
const runtimePorts = resolveRuntimePorts();
const testServerEnv = {
...sanitizeColorEnv(process.env),
OMNIROUTE_DISABLE_TOKEN_HEALTHCHECK: process.env.OMNIROUTE_DISABLE_TOKEN_HEALTHCHECK || "1",
OMNIROUTE_HIDE_HEALTHCHECK_LOGS: process.env.OMNIROUTE_HIDE_HEALTHCHECK_LOGS || "1",
};
const args = [
"./node_modules/next/dist/bin/next",
mode,
@@ -66,5 +72,5 @@ if (mode === "dev") {
spawnWithForwardedSignals(process.execPath, args, {
stdio: "inherit",
env: withRuntimePortEnv(process.env, runtimePorts),
env: withRuntimePortEnv(testServerEnv, runtimePorts),
});
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env node
import { spawn } from "node:child_process";
import { sanitizeColorEnv } from "./runtime-env.mjs";
const defaultArgs = ["test", "tests/e2e/*.spec.ts"];
const forwardedArgs = process.argv.slice(2);
const args = forwardedArgs.length > 0 ? forwardedArgs : defaultArgs;
const playwrightEnv = sanitizeColorEnv(process.env);
delete playwrightEnv.NO_COLOR;
delete playwrightEnv.FORCE_COLOR;
const child = spawn(process.execPath, ["./node_modules/playwright/cli.js", ...args], {
stdio: "inherit",
env: playwrightEnv,
});
child.on("exit", (code, signal) => {
if (signal) {
process.kill(process.pid, signal);
return;
}
process.exit(code ?? 0);
});
+12
View File
@@ -25,6 +25,18 @@ export function withRuntimePortEnv(env, runtimePorts) {
};
}
export function sanitizeColorEnv(env = {}) {
const sanitized = { ...env };
// Node warns when both FORCE_COLOR and NO_COLOR are set.
// Prefer NO_COLOR in test tooling to avoid noisy process warnings.
if (typeof sanitized.FORCE_COLOR !== "undefined" && typeof sanitized.NO_COLOR !== "undefined") {
delete sanitized.FORCE_COLOR;
}
return sanitized;
}
export function spawnWithForwardedSignals(command, args, options = {}) {
const child = spawn(command, args, options);
@@ -168,6 +168,33 @@ describe("API Routes — export HTTP methods", () => {
});
});
describe("API Routes — T09 /v1 catalog consistency", () => {
const v1RouteSrc = readProjectFile("src/app/api/v1/route.ts");
const v1ModelsRouteSrc = readProjectFile("src/app/api/v1/models/route.ts");
const v1CatalogSrc = readProjectFile("src/app/api/v1/models/catalog.ts");
it("/api/v1 should delegate model catalog to unified builder", () => {
assert.ok(v1RouteSrc, "src/app/api/v1/route.ts should exist");
assert.match(v1RouteSrc, /getUnifiedModelsResponse/);
assert.match(v1RouteSrc, /from\s+["']\.\/models\/catalog["']/);
assert.doesNotMatch(v1RouteSrc, /const\s+models\s*=\s*\[/);
});
it("/api/v1/models route should only consume unified model catalog builder", () => {
assert.ok(v1ModelsRouteSrc, "src/app/api/v1/models/route.ts should exist");
assert.match(v1ModelsRouteSrc, /from\s+["']\.\/catalog["']/);
assert.doesNotMatch(
v1ModelsRouteSrc,
/export\s+async\s+function\s+getUnifiedModelsResponse\s*\(/
);
});
it("/api/v1/models/catalog should export unified model catalog builder", () => {
assert.ok(v1CatalogSrc, "src/app/api/v1/models/catalog.ts should exist");
assert.match(v1CatalogSrc, /export\s+async\s+function\s+getUnifiedModelsResponse\s*\(/);
});
});
// ─── Barrel Exports ─────────────────────────────────
describe("Barrel Exports — shared/components", () => {
@@ -149,3 +149,106 @@ test("server-init.ts calls enforceSecrets", () => {
assert.ok(content, "src/server-init.ts should exist");
assert.ok(content.includes("enforceSecrets"), "server-init.ts should call enforceSecrets");
});
// ─── T06/T07 Regression Checks ───────────────────────
test("callLogs.ts wires no-log and PII sanitization before persistence", () => {
const content = readIfExists("src/lib/usage/callLogs.ts");
assert.ok(content, "src/lib/usage/callLogs.ts should exist");
assert.ok(
content.includes('from "../compliance"'),
"callLogs.ts should import compliance module"
);
assert.ok(content.includes('from "../piiSanitizer"'), "callLogs.ts should import piiSanitizer");
assert.ok(content.includes("isNoLog("), "callLogs.ts should check no-log policy");
assert.ok(content.includes("sanitizePayloadPII"), "callLogs.ts should sanitize PII recursively");
});
test("API key update route and DB layer wire persisted no-log controls", () => {
const routeContent = readIfExists("src/app/api/keys/[id]/route.ts");
assert.ok(routeContent, "src/app/api/keys/[id]/route.ts should exist");
assert.ok(routeContent.includes("noLog"), "key PATCH route should handle noLog field");
const dbContent = readIfExists("src/lib/db/apiKeys.ts");
assert.ok(dbContent, "src/lib/db/apiKeys.ts should exist");
assert.ok(dbContent.includes("no_log"), "api key DB module should persist no_log column");
});
test("MCP server enforces scopes from caller context before tool execution", () => {
const serverContent = readIfExists("open-sse/mcp-server/server.ts");
assert.ok(serverContent, "open-sse/mcp-server/server.ts should exist");
assert.ok(
serverContent.includes("resolveCallerScopeContext"),
"MCP server should resolve caller scopes from request context"
);
assert.ok(
serverContent.includes("evaluateToolScopes"),
"MCP server should evaluate required scopes per tool"
);
const scopeContent = readIfExists("open-sse/mcp-server/scopeEnforcement.ts");
assert.ok(scopeContent, "open-sse/mcp-server/scopeEnforcement.ts should exist");
assert.ok(
scopeContent.includes("authInfo"),
"scope enforcement should parse authInfo scopes when provided by transport"
);
});
test("T06 route payload validation uses validateBody in critical endpoints", () => {
const targets = [
"src/app/api/usage/budget/route.ts",
"src/app/api/policies/route.ts",
"src/app/api/fallback/chains/route.ts",
"src/app/api/models/route.ts",
"src/app/api/models/availability/route.ts",
"src/app/api/provider-models/route.ts",
"src/app/api/pricing/route.ts",
"src/app/api/rate-limits/route.ts",
"src/app/api/resilience/route.ts",
"src/app/api/v1/embeddings/route.ts",
"src/app/api/v1/images/generations/route.ts",
"src/app/api/v1/audio/speech/route.ts",
"src/app/api/v1/moderations/route.ts",
"src/app/api/v1/rerank/route.ts",
"src/app/api/oauth/[provider]/[action]/route.ts",
"src/app/api/oauth/cursor/import/route.ts",
"src/app/api/oauth/kiro/import/route.ts",
"src/app/api/oauth/kiro/social-exchange/route.ts",
"src/app/api/cloud/credentials/update/route.ts",
"src/app/api/cloud/model/resolve/route.ts",
"src/app/api/cloud/models/alias/route.ts",
"src/app/api/sync/cloud/route.ts",
"src/app/api/combos/[id]/route.ts",
"src/app/api/combos/test/route.ts",
"src/app/api/db-backups/route.ts",
"src/app/api/evals/route.ts",
"src/app/api/keys/[id]/route.ts",
"src/app/api/models/alias/route.ts",
"src/app/api/provider-nodes/route.ts",
"src/app/api/provider-nodes/[id]/route.ts",
"src/app/api/provider-nodes/validate/route.ts",
"src/app/api/providers/[id]/route.ts",
"src/app/api/providers/test-batch/route.ts",
"src/app/api/providers/validate/route.ts",
"src/app/api/v1beta/models/[...path]/route.ts",
"src/app/api/cli-tools/antigravity-mitm/route.ts",
"src/app/api/cli-tools/antigravity-mitm/alias/route.ts",
"src/app/api/cli-tools/backups/route.ts",
"src/app/api/cli-tools/claude-settings/route.ts",
"src/app/api/cli-tools/cline-settings/route.ts",
"src/app/api/cli-tools/codex-profiles/route.ts",
"src/app/api/cli-tools/codex-settings/route.ts",
"src/app/api/cli-tools/droid-settings/route.ts",
"src/app/api/cli-tools/guide-settings/[toolId]/route.ts",
"src/app/api/cli-tools/kilo-settings/route.ts",
"src/app/api/cli-tools/openclaw-settings/route.ts",
];
for (const relPath of targets) {
const content = readIfExists(relPath);
assert.ok(content, `${relPath} should exist`);
assert.ok(
content.includes("validateBody("),
`${relPath} should validate payload with validateBody`
);
}
});
@@ -0,0 +1,171 @@
import test from "node:test";
import assert from "node:assert/strict";
const BASE_URL = "http://localhost:20128";
test("contract: /api/v1 OPTIONS exposes CORS and allowed methods", async () => {
const { OPTIONS } = await import("../../src/app/api/v1/route.ts");
const response = await OPTIONS();
assert.equal(response.status, 200);
assert.ok(response.headers.has("Access-Control-Allow-Origin"));
});
test("contract: /api/v1/embeddings OPTIONS exposes POST/GET/OPTIONS", async () => {
const { OPTIONS } = await import("../../src/app/api/v1/embeddings/route.ts");
const response = await OPTIONS();
const allowMethods = response.headers.get("Access-Control-Allow-Methods") || "";
assert.equal(response.status, 200);
assert.ok(allowMethods.includes("GET"));
assert.ok(allowMethods.includes("POST"));
assert.ok(allowMethods.includes("OPTIONS"));
});
test("contract: /api/v1 and /api/v1/models return consistent model IDs", async () => {
const [{ GET: getV1 }, { GET: getV1Models }] = await Promise.all([
import("../../src/app/api/v1/route.ts"),
import("../../src/app/api/v1/models/route.ts"),
]);
const [v1Response, v1ModelsResponse] = await Promise.all([
getV1(new Request(`${BASE_URL}/api/v1`, { method: "GET" })),
getV1Models(new Request(`${BASE_URL}/api/v1/models`, { method: "GET" })),
]);
assert.equal(v1Response.status, 200);
assert.equal(v1ModelsResponse.status, 200);
const v1Body = await v1Response.json();
const v1ModelsBody = await v1ModelsResponse.json();
assert.equal(v1Body.object, "list");
assert.equal(v1ModelsBody.object, "list");
assert.ok(Array.isArray(v1Body.data));
assert.ok(Array.isArray(v1ModelsBody.data));
const v1Ids = [...new Set(v1Body.data.map((item) => item.id))].sort();
const v1ModelsIds = [...new Set(v1ModelsBody.data.map((item) => item.id))].sort();
assert.deepEqual(v1Ids, v1ModelsIds);
});
test("contract: /api/v1/models returns OpenAI-compatible model shape", async () => {
const { GET: getV1Models } = await import("../../src/app/api/v1/models/route.ts");
const response = await getV1Models(new Request(`${BASE_URL}/api/v1/models`, { method: "GET" }));
assert.equal(response.status, 200);
const body = await response.json();
assert.equal(body.object, "list");
assert.ok(Array.isArray(body.data));
assert.ok(body.data.length > 0, "models list should not be empty");
const first = body.data[0];
assert.equal(typeof first.id, "string");
assert.equal(first.object, "model");
assert.equal(typeof first.created, "number");
assert.equal(typeof first.owned_by, "string");
});
test("contract: /api/v1/embeddings GET returns embedding model listing shape", async () => {
const { GET: getEmbeddings } = await import("../../src/app/api/v1/embeddings/route.ts");
const response = await getEmbeddings();
assert.equal(response.status, 200);
const body = await response.json();
assert.equal(body.object, "list");
assert.ok(Array.isArray(body.data));
assert.ok(body.data.length > 0, "embedding model list should not be empty");
const first = body.data[0];
assert.equal(first.object, "model");
assert.equal(first.type, "embedding");
assert.equal(typeof first.id, "string");
assert.equal(typeof first.owned_by, "string");
});
test("contract: /api/v1/images/generations GET returns image model listing shape", async () => {
const { GET: getImageModels } = await import("../../src/app/api/v1/images/generations/route.ts");
const response = await getImageModels();
assert.equal(response.status, 200);
const body = await response.json();
assert.equal(body.object, "list");
assert.ok(Array.isArray(body.data));
assert.ok(body.data.length > 0, "image model list should not be empty");
const first = body.data[0];
assert.equal(first.object, "model");
assert.equal(first.type, "image");
assert.equal(typeof first.id, "string");
assert.equal(typeof first.owned_by, "string");
});
test("contract: /api/v1/messages/count_tokens returns 400 on invalid JSON", async () => {
const { POST: countTokens } = await import("../../src/app/api/v1/messages/count_tokens/route.ts");
const response = await countTokens(
new Request(`${BASE_URL}/api/v1/messages/count_tokens`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: "not-json",
})
);
assert.equal(response.status, 400);
const body = await response.json();
assert.ok(body.error, "error payload should exist");
assert.ok(
typeof body.error === "string" || typeof body.error === "object",
"error payload should be string or object"
);
});
test("contract: /api/v1/messages/count_tokens rejects empty messages payload", async () => {
const { POST: countTokens } = await import("../../src/app/api/v1/messages/count_tokens/route.ts");
const response = await countTokens(
new Request(`${BASE_URL}/api/v1/messages/count_tokens`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ messages: [] }),
})
);
assert.equal(response.status, 400);
const body = await response.json();
assert.ok(body.error, "error payload should exist");
assert.ok(
typeof body.error === "string" || typeof body.error === "object",
"error payload should be string or object"
);
});
test("contract: /api/v1/messages/count_tokens computes token estimate from text content", async () => {
const { POST: countTokens } = await import("../../src/app/api/v1/messages/count_tokens/route.ts");
const payload = {
messages: [
{ role: "user", content: "abcd" }, // 4 chars
{
role: "assistant",
content: [{ type: "text", text: "12345678" }], // 8 chars
},
],
};
const response = await countTokens(
new Request(`${BASE_URL}/api/v1/messages/count_tokens`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(payload),
})
);
assert.equal(response.status, 200);
const body = await response.json();
assert.equal(body.input_tokens, 3);
});
+7 -9
View File
@@ -39,9 +39,9 @@ export const options = {
executor: "ramping-vus",
startVUs: 1,
stages: [
{ duration: "10s", target: VUS }, // Ramp up
{ duration: DURATION, target: VUS }, // Sustained load
{ duration: "10s", target: 0 }, // Ramp down
{ duration: "10s", target: VUS }, // Ramp up
{ duration: DURATION, target: VUS }, // Sustained load
{ duration: "10s", target: 0 }, // Ramp down
],
exec: "chatCompletions",
},
@@ -54,8 +54,8 @@ export const options = {
},
},
thresholds: {
http_req_duration: ["p(95)<5000"], // 95% of requests < 5s
errors: ["rate<0.1"], // Error rate < 10%
http_req_duration: ["p(95)<5000"], // 95% of requests < 5s
errors: ["rate<0.1"], // Error rate < 10%
chat_latency: ["p(50)<3000", "p(95)<8000"],
health_latency: ["p(95)<500"],
},
@@ -76,9 +76,7 @@ const headers = {
export function chatCompletions() {
const payload = JSON.stringify({
model: "gpt-4o-mini",
messages: [
{ role: "user", content: "Say hello in one word." },
],
messages: [{ role: "user", content: "Say hello in one word." }],
temperature: 0,
max_tokens: 10,
stream: false,
@@ -112,7 +110,7 @@ export function chatCompletions() {
* Health Check lightweight endpoint to measure base latency.
*/
export function healthCheck() {
const res = http.get(`${BASE_URL}/api/health`, {
const res = http.get(`${BASE_URL}/api/monitoring/health`, {
headers: { Authorization: `Bearer ${API_KEY}` },
timeout: "5s",
});
+132
View File
@@ -0,0 +1,132 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
validateBody,
translatorDetectSchema,
translatorSaveSchema,
translatorSendSchema,
translatorTranslateSchema,
cliSettingsEnvSchema,
v1EmbeddingsSchema,
providerChatCompletionSchema,
v1CountTokensSchema,
} from "../../src/shared/validation/schemas.ts";
test("translatorDetectSchema rejects empty body object", () => {
const validation = validateBody(translatorDetectSchema, { body: {} });
assert.equal(validation.success, false);
});
test("translatorSendSchema rejects empty body object", () => {
const validation = validateBody(translatorSendSchema, {
provider: "openai",
body: {},
});
assert.equal(validation.success, false);
});
test("translatorSaveSchema rejects unsupported file name", () => {
const validation = validateBody(translatorSaveSchema, {
file: "random.txt",
content: "ok",
});
assert.equal(validation.success, false);
});
test("translatorSaveSchema rejects non-string content", () => {
const validation = validateBody(translatorSaveSchema, {
file: "1_req_client.json",
content: { raw: true },
});
assert.equal(validation.success, false);
});
test("translatorTranslateSchema requires explicit step", () => {
const validation = validateBody(translatorTranslateSchema, {
provider: "openai",
body: { model: "gpt-4o-mini" },
});
assert.equal(validation.success, false);
});
test("translatorTranslateSchema requires provider for non-direct step", () => {
const validation = validateBody(translatorTranslateSchema, {
step: 2,
body: { model: "gpt-4o-mini" },
});
assert.equal(validation.success, false);
});
test("cliSettingsEnvSchema coerces numeric and boolean values to string", () => {
const validation = validateBody(cliSettingsEnvSchema, {
env: {
API_TIMEOUT_MS: 60000,
ANTHROPIC_USE_PROXY: true,
},
});
assert.equal(validation.success, true);
if (validation.success) {
assert.equal(validation.data.env.API_TIMEOUT_MS, "60000");
assert.equal(validation.data.env.ANTHROPIC_USE_PROXY, "true");
}
});
test("cliSettingsEnvSchema rejects invalid key format", () => {
const validation = validateBody(cliSettingsEnvSchema, {
env: {
"anthropic-base-url": "https://example.com/v1",
},
});
assert.equal(validation.success, false);
});
test("v1EmbeddingsSchema accepts string and token-array inputs", () => {
const withString = validateBody(v1EmbeddingsSchema, {
model: "openai/text-embedding-3-small",
input: "hello world",
});
assert.equal(withString.success, true);
const withTokenArray = validateBody(v1EmbeddingsSchema, {
model: "openai/text-embedding-3-small",
input: [101, 102, 103],
});
assert.equal(withTokenArray.success, true);
});
test("v1EmbeddingsSchema rejects empty embedding input", () => {
const validation = validateBody(v1EmbeddingsSchema, {
model: "openai/text-embedding-3-small",
input: [],
});
assert.equal(validation.success, false);
});
test("providerChatCompletionSchema requires model", () => {
const validation = validateBody(providerChatCompletionSchema, {
messages: [{ role: "user", content: "hello" }],
});
assert.equal(validation.success, false);
});
test("providerChatCompletionSchema requires at least one message/input/prompt field", () => {
const validation = validateBody(providerChatCompletionSchema, {
model: "openai/gpt-4o-mini",
});
assert.equal(validation.success, false);
});
test("providerChatCompletionSchema accepts valid message payload", () => {
const validation = validateBody(providerChatCompletionSchema, {
model: "openai/gpt-4o-mini",
messages: [{ role: "user", content: "hello" }],
});
assert.equal(validation.success, true);
});
test("v1CountTokensSchema rejects empty messages", () => {
const validation = validateBody(v1CountTokensSchema, {
messages: [],
});
assert.equal(validation.success, false);
});
+138
View File
@@ -0,0 +1,138 @@
import test from "node:test";
import assert from "node:assert/strict";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
const TEST_DATA_DIR = fs.mkdtempSync(path.join(os.tmpdir(), "omniroute-t07-"));
process.env.DATA_DIR = TEST_DATA_DIR;
const originalPiiEnabled = process.env.PII_RESPONSE_SANITIZATION;
const originalPiiMode = process.env.PII_RESPONSE_SANITIZATION_MODE;
const originalApiKeySecret = process.env.API_KEY_SECRET;
process.env.PII_RESPONSE_SANITIZATION = "true";
process.env.PII_RESPONSE_SANITIZATION_MODE = "redact";
process.env.API_KEY_SECRET = process.env.API_KEY_SECRET || "t07-test-secret-key";
const core = await import("../../src/lib/db/core.ts");
const apiKeysDb = await import("../../src/lib/db/apiKeys.ts");
const compliance = await import("../../src/lib/compliance/index.ts");
const callLogs = await import("../../src/lib/usage/callLogs.ts");
const schemas = await import("../../src/shared/validation/schemas.ts");
async function resetStorage() {
core.resetDbInstance();
apiKeysDb.resetApiKeyState();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
fs.mkdirSync(TEST_DATA_DIR, { recursive: true });
}
test.beforeEach(async () => {
await resetStorage();
});
test.after(async () => {
core.resetDbInstance();
apiKeysDb.resetApiKeyState();
fs.rmSync(TEST_DATA_DIR, { recursive: true, force: true });
if (originalPiiEnabled === undefined) {
delete process.env.PII_RESPONSE_SANITIZATION;
} else {
process.env.PII_RESPONSE_SANITIZATION = originalPiiEnabled;
}
if (originalPiiMode === undefined) {
delete process.env.PII_RESPONSE_SANITIZATION_MODE;
} else {
process.env.PII_RESPONSE_SANITIZATION_MODE = originalPiiMode;
}
if (originalApiKeySecret === undefined) {
delete process.env.API_KEY_SECRET;
} else {
process.env.API_KEY_SECRET = originalApiKeySecret;
}
});
test("updateKeyPermissionsSchema accepts noLog-only updates and rejects empty payload", () => {
const noLogOnly = schemas.validateBody(schemas.updateKeyPermissionsSchema, { noLog: true });
assert.equal(noLogOnly.success, true);
const emptyPayload = schemas.validateBody(schemas.updateKeyPermissionsSchema, {});
assert.equal(emptyPayload.success, false);
});
test("API key no_log persists and updates compliance state", async () => {
const created = await apiKeysDb.createApiKey("privacy-key", "machine-test");
const initial = await apiKeysDb.getApiKeyById(created.id);
assert.equal(initial?.noLog, false);
assert.equal(compliance.isNoLog(created.id), false);
const updated = await apiKeysDb.updateApiKeyPermissions(created.id, { noLog: true });
assert.equal(updated, true);
const afterEnable = await apiKeysDb.getApiKeyById(created.id);
assert.equal(afterEnable?.noLog, true);
const metadata = await apiKeysDb.getApiKeyMetadata(created.key);
assert.equal(metadata?.noLog, true);
assert.equal(compliance.isNoLog(created.id), true);
const reverted = await apiKeysDb.updateApiKeyPermissions(created.id, { noLog: false });
assert.equal(reverted, true);
assert.equal(compliance.isNoLog(created.id), false);
});
test("call logs omit payloads when key no_log is enabled and redact PII otherwise", async () => {
const created = await apiKeysDb.createApiKey("privacy-log-key", "machine-test");
const baseEntry = {
method: "POST",
path: "/v1/chat/completions",
status: 200,
model: "openai/gpt-4.1",
provider: "openai",
duration: 42,
apiKeyId: created.id,
apiKeyName: created.name,
requestBody: {
email: "john@example.com",
token: "super-secret-token",
nested: {
contact: "john@example.com",
},
},
responseBody: {
summary: "Contact john@example.com for details",
},
};
await apiKeysDb.updateApiKeyPermissions(created.id, { noLog: true });
await callLogs.saveCallLog(baseEntry);
const firstBatch = await callLogs.getCallLogs({ limit: 5 });
assert.equal(firstBatch.length, 1);
assert.equal(firstBatch[0].hasRequestBody, false);
assert.equal(firstBatch[0].hasResponseBody, false);
const noLogDetails = await callLogs.getCallLogById(firstBatch[0].id);
assert.equal(noLogDetails?.requestBody, null);
assert.equal(noLogDetails?.responseBody, null);
await apiKeysDb.updateApiKeyPermissions(created.id, { noLog: false });
await callLogs.saveCallLog(baseEntry);
const secondBatch = await callLogs.getCallLogs({ limit: 10 });
assert.equal(secondBatch.length, 2);
const withPayloadEntry = secondBatch.find((item) => item.id !== firstBatch[0].id);
assert.ok(withPayloadEntry, "Expected a log entry with payload persisted");
const payloadDetails = await callLogs.getCallLogById(withPayloadEntry.id);
assert.equal(payloadDetails?.requestBody?.email, "[EMAIL_REDACTED]");
assert.equal(payloadDetails?.requestBody?.token, "[REDACTED]");
assert.equal(payloadDetails?.requestBody?.nested?.contact, "[EMAIL_REDACTED]");
assert.equal(payloadDetails?.responseBody?.summary, "Contact [EMAIL_REDACTED] for details");
});
@@ -0,0 +1,72 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
evaluateToolScopes,
resolveCallerScopeContext,
} from "../../open-sse/mcp-server/scopeEnforcement.ts";
test("resolveCallerScopeContext prioritizes authInfo scopes", () => {
const context = resolveCallerScopeContext(
{
authInfo: {
clientId: "client-auth",
scopes: ["read:health", "read:combos"],
},
_meta: { scopes: ["write:combos"] },
sessionId: "session-1",
},
["read:usage"]
);
assert.equal(context.callerId, "client-auth");
assert.equal(context.source, "authInfo");
assert.deepEqual(context.scopes, ["read:health", "read:combos"]);
});
test("resolveCallerScopeContext falls back to _meta scopes", () => {
const context = resolveCallerScopeContext(
{
_meta: {
scopes: ["read:quota", "read:models"],
},
sessionId: "session-meta",
},
["read:usage"]
);
assert.equal(context.callerId, "session-meta");
assert.equal(context.source, "meta");
assert.deepEqual(context.scopes, ["read:quota", "read:models"]);
});
test("resolveCallerScopeContext uses env fallback when caller has no scopes", () => {
const context = resolveCallerScopeContext({ sessionId: "session-env" }, ["read:health"]);
assert.equal(context.source, "env");
assert.deepEqual(context.scopes, ["read:health"]);
});
test("evaluateToolScopes allows requests when enforcement is disabled", () => {
const check = evaluateToolScopes("omniroute_switch_combo", [], false);
assert.equal(check.allowed, true);
assert.deepEqual(check.missing, []);
});
test("evaluateToolScopes denies tool execution when required scope is missing", () => {
const check = evaluateToolScopes("omniroute_switch_combo", ["read:combos"], true);
assert.equal(check.allowed, false);
assert.ok(check.missing.includes("write:combos"));
assert.equal(check.reason, "missing_scopes");
});
test("evaluateToolScopes supports wildcard scopes", () => {
const check = evaluateToolScopes("omniroute_get_health", ["read:*"], true);
assert.equal(check.allowed, true);
assert.deepEqual(check.missing, []);
});
test("evaluateToolScopes denies unknown tool names", () => {
const check = evaluateToolScopes("omniroute_unknown_tool", ["*"], true);
assert.equal(check.allowed, false);
assert.equal(check.reason, "tool_definition_missing");
});
+9 -1
View File
@@ -36,5 +36,13 @@
".next/types/**/*.ts",
".next/dev/types/**/*.ts"
],
"exclude": ["node_modules", "open-sse", "antigravity-manager-analysis"]
"exclude": [
"node_modules",
"open-sse",
"antigravity-manager-analysis",
"app.__qa_backup",
"vscode-extension",
"omnirouteCloud",
"omnirouteSite"
]
}
+23
View File
@@ -0,0 +1,23 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"strict": false,
"strictNullChecks": true,
"noImplicitAny": false,
"noEmit": true,
"incremental": false
},
"include": [],
"files": [
"src/app/api/settings/proxy/test/route.ts",
"src/lib/db/stateReset.ts",
"src/shared/validation/providerSchema.ts",
"src/shared/validation/schemas.ts",
"open-sse/config/providerModels.ts",
"open-sse/config/providerRegistry.ts",
"open-sse/mcp-server/server.ts",
"open-sse/mcp-server/scopeEnforcement.ts",
"open-sse/translator/registry.ts"
],
"exclude": ["node_modules", ".next", "app.__qa_backup", "vscode-extension"]
}
+19
View File
@@ -0,0 +1,19 @@
{
"extends": "./tsconfig.json",
"compilerOptions": {
"strict": false,
"strictNullChecks": true,
"noImplicitAny": true,
"noEmit": true,
"incremental": false
},
"include": [],
"files": [
"src/lib/db/stateReset.ts",
"open-sse/config/providerModels.ts",
"open-sse/mcp-server/server.ts",
"open-sse/translator/registry.ts",
"open-sse/mcp-server/scopeEnforcement.ts"
],
"exclude": ["node_modules", ".next", "app.__qa_backup", "vscode-extension"]
}