Compare commits

...

2 Commits

Author SHA1 Message Date
diegosouzapw 1b354be827 feat: T07 — API Key Round-Robin per provider connection
Build Electron Desktop App / Validate version (push) Failing after 42s
Build Electron Desktop App / Build Electron (macos-arm64) (push) Has been skipped
Build Electron Desktop App / Build Electron (linux) (push) Has been skipped
Build Electron Desktop App / Build Electron (macos-intel) (push) Has been skipped
Build Electron Desktop App / Build Electron (windows) (push) Has been skipped
Build Electron Desktop App / Create Release (push) Has been skipped
- New: open-sse/services/apiKeyRotator.ts — round-robin rotation
  between primary API key + providerSpecificData.extraApiKeys[]
- Modified: open-sse/executors/base.ts — buildHeaders() rotates key
  using getRotatingApiKey() when extraApiKeys configured
- Modified: open-sse/handlers/chatCore.ts — injects connectionId into
  credentials to enable per-connection rotation index tracking
- Modified: providers/[id]/page.tsx — 'Extra API Keys' UI section in
  EditConnectionModal: add/remove keys, persisted in providerSpecificData

T08 (quota window rolling) and T13 (wildcard model routing) confirmed
already implemented in accountFallback.ts and wildcardRouter.ts.
2026-03-14 15:03:54 -03:00
diegosouzapw 75a6d850fc chore: release v2.4.3
Build Electron Desktop App / Validate version (push) Failing after 31s
Build Electron Desktop App / Build Electron (macos-arm64) (push) Has been skipped
Build Electron Desktop App / Build Electron (linux) (push) Has been skipped
Build Electron Desktop App / Build Electron (macos-intel) (push) Has been skipped
Build Electron Desktop App / Build Electron (windows) (push) Has been skipped
Build Electron Desktop App / Create Release (push) Has been skipped
- fix: Codex/GitHub limits page HTTP 500 → graceful 401/403 messages
- fix: MaintenanceBanner false-positive on page load (stale closure)
- fix: add title tooltips to edit/delete buttons in ConnectionCard
- feat: add fill-first and p2c routing strategies to combo picker
- feat: Free Stack template pre-fills 7 free provider models
- feat: combo create/edit modal wider (max-w-4xl)
2026-03-14 12:49:36 -03:00
11 changed files with 276 additions and 27 deletions
+26 -1
View File
@@ -2,7 +2,32 @@
## [Unreleased]
## [2.4.2] - 2026-03-14
## [2.4.4] - 2026-03-14
> API Key Round-Robin support for multi-key provider setups, and confirmation of wildcard routing and quota window rolling already in place.
### ✨ New Features
- **API Key Round-Robin (T07)**: Provider connections can now hold multiple API keys (Edit Connection → Extra API Keys). Requests rotate round-robin between primary + extra keys via `providerSpecificData.extraApiKeys[]`. Keys are held in-memory indexed per connection — no DB schema changes required.
### 📝 Already Implemented (confirmed in audit)
- **Wildcard Model Routing (T13)**: `wildcardRouter.ts` with glob-style wildcard matching (`gpt*`, `claude-?-sonnet`, etc.) is already integrated into `model.ts` with specificity ranking.
- **Quota Window Rolling (T08)**: `accountFallback.ts:isModelLocked()` already auto-advances the window — if `Date.now() > entry.until`, lock is deleted immediately (no stale blocking).
> UI polish, routing strategy additions, and graceful error handling for usage limits.
### ✨ New Features
- **Fill-First & P2C Routing Strategies**: Added `fill-first` (drain quota before moving on) and `p2c` (Power-of-Two-Choices low-latency selection) to combo strategy picker, with full guidance panels and color-coded badges.
- **Free Stack Preset Models**: Creating a combo with the Free Stack template now auto-fills 7 best-in-class free provider models (Gemini CLI, Kiro, iFlow×2, Qwen, NVIDIA NIM, Groq). Users just activate the providers and get a $0/month combo out-of-the-box.
- **Wider Combo Modal**: Create/Edit combo modal now uses `max-w-4xl` for comfortable editing of large combos.
### 🐛 Bug Fixes
- **Limits page HTTP 500 for Codex & GitHub**: `getCodexUsage()` and `getGitHubUsage()` now return a user-friendly message when the provider returns 401/403 (expired token), instead of throwing and causing a 500 error on the Limits page.
- **MaintenanceBanner false-positive**: Banner no longer shows "Server is unreachable" spuriously on page load. Fixed by calling `checkHealth()` immediately on mount and removing stale `show`-state closure.
- **Provider icon tooltips**: Edit (pencil) and delete icon buttons in the provider connection row now have native HTML tooltips — all 6 action icons are now self-documented.
> Multiple improvements from community issue analysis, new provider support, bug fixes for token tracking, model routing, and streaming reliability.
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.1.0
info:
title: OmniRoute API
version: 2.4.2
version: 2.4.4
description: |
OmniRoute is a local-first AI API proxy router. It provides an OpenAI-compatible
endpoint that routes requests to multiple AI providers with load balancing,
+10 -1
View File
@@ -1,5 +1,6 @@
import { HTTP_STATUS, FETCH_TIMEOUT_MS } from "../config/constants.ts";
import { applyFingerprint, isCliCompatEnabled } from "../config/cliFingerprints.ts";
import { getRotatingApiKey } from "../services/apiKeyRotator.ts";
type JsonRecord = Record<string, unknown>;
@@ -23,6 +24,7 @@ export type ProviderCredentials = {
refreshToken?: string;
apiKey?: string;
expiresAt?: string;
connectionId?: string; // T07: used for API key rotation index
providerSpecificData?: JsonRecord;
};
@@ -131,7 +133,14 @@ export class BaseExecutor {
if (credentials.accessToken) {
headers["Authorization"] = `Bearer ${credentials.accessToken}`;
} else if (credentials.apiKey) {
headers["Authorization"] = `Bearer ${credentials.apiKey}`;
// T07: rotate between primary + extra API keys when extraApiKeys is configured
const extraKeys =
(credentials.providerSpecificData?.extraApiKeys as string[] | undefined) ?? [];
const effectiveKey =
extraKeys.length > 0 && credentials.connectionId
? getRotatingApiKey(credentials.connectionId, credentials.apiKey, extraKeys)
: credentials.apiKey;
headers["Authorization"] = `Bearer ${effectiveKey}`;
}
if (stream) {
+6
View File
@@ -94,6 +94,12 @@ export async function handleChatCore({
// Initialize rate limit settings from persisted DB (once, lazy)
await initializeRateLimits();
// T07: Inject connectionId into credentials so executors can rotate API keys
// using providerSpecificData.extraApiKeys (API Key Round-Robin feature)
if (connectionId && credentials && !credentials.connectionId) {
credentials.connectionId = connectionId;
}
const sourceFormat = detectFormat(body);
const endpointPath = (clientRawRequest?.endpoint || "").toLowerCase();
const isResponsesEndpoint = endpointPath.endsWith("/responses");
+63
View File
@@ -0,0 +1,63 @@
/**
* apiKeyRotator.ts — T07: API Key Round-Robin
*
* Rotates between a primary API key and extra API keys stored in
* providerSpecificData.extraApiKeys[]. Uses round-robin by default.
*
* Extra keys are stored as plain strings in providerSpecificData.extraApiKeys.
* Example: { extraApiKeys: ["sk-abc...", "sk-def...", "sk-ghi..."] }
*
* The in-memory rotation index resets on process restart, which is intentional —
* it ensures even distribution across restarts without persistence overhead.
*/
// In-memory round-robin index per connection
const _keyIndexes = new Map<string, number>();
/**
* Get the next API key in round-robin rotation for a given connection.
* If no extra keys are configured, returns the primary key unchanged.
*
* @param connectionId - Unique connection identifier (for index isolation)
* @param primaryKey - The main api_key from the connection
* @param extraKeys - Additional API keys from providerSpecificData.extraApiKeys
* @returns The selected API key (may be primary or one of the extras)
*/
export function getRotatingApiKey(
connectionId: string,
primaryKey: string,
extraKeys: string[] = []
): string {
const validExtras = extraKeys.filter((k) => typeof k === "string" && k.trim().length > 0);
// Only 1 key available → no rotation needed
if (validExtras.length === 0) return primaryKey;
const allKeys = [primaryKey, ...validExtras].filter(Boolean);
if (allKeys.length <= 1) return primaryKey;
const current = _keyIndexes.get(connectionId) ?? 0;
const idx = current % allKeys.length;
_keyIndexes.set(connectionId, current + 1);
return allKeys[idx];
}
/**
* Reset the rotation index for a connection.
* Call this when a key fails (401/403) to skip the bad key next time.
*
* @param connectionId - Connection to reset
*/
export function resetRotationIndex(connectionId: string): void {
_keyIndexes.delete(connectionId);
}
/**
* Get the total number of API keys available for a connection.
* Used for logging/observability.
*/
export function getApiKeyCount(primaryKey: string, extraKeys: string[] = []): number {
const validExtras = extraKeys.filter((k) => typeof k === "string" && k.trim().length > 0);
return (primaryKey ? 1 : 0) + validExtras.length;
}
+10
View File
@@ -161,6 +161,11 @@ async function getGitHubUsage(accessToken, providerSpecificData) {
if (!response.ok) {
const error = await response.text();
if (response.status === 401 || response.status === 403) {
return {
message: `GitHub token expired or permission denied. Please re-authenticate the connection.`,
};
}
throw new Error(`GitHub API error: ${error}`);
}
@@ -620,6 +625,11 @@ async function getCodexUsage(accessToken, providerSpecificData: Record<string, u
});
if (!response.ok) {
if (response.status === 401 || response.status === 403) {
return {
message: `Codex token expired or access denied. Please re-authenticate the connection.`,
};
}
throw new Error(`Codex API error: ${response.status}`);
}
+3 -2
View File
@@ -1,12 +1,12 @@
{
"name": "omniroute",
"version": "2.4.1",
"version": "2.4.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "omniroute",
"version": "2.4.1",
"version": "2.4.3",
"hasInstallScript": true,
"license": "MIT",
"workspaces": [
@@ -6866,6 +6866,7 @@
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "omniroute",
"version": "2.4.2",
"version": "2.4.4",
"description": "Smart AI Router with auto fallback — route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
"type": "module",
"bin": {
+58 -1
View File
@@ -27,6 +27,13 @@ const STRATEGY_OPTIONS = [
{ value: "random", labelKey: "random", descKey: "randomDesc", icon: "shuffle" },
{ value: "least-used", labelKey: "leastUsed", descKey: "leastUsedDesc", icon: "low_priority" },
{ value: "cost-optimized", labelKey: "costOpt", descKey: "costOptimizedDesc", icon: "savings" },
{
value: "fill-first",
labelKey: "fillFirst",
descKey: "fillFirstDesc",
icon: "stacked_bar_chart",
},
{ value: "p2c", labelKey: "p2c", descKey: "p2cDesc", icon: "compare_arrows" },
];
const STRATEGY_GUIDANCE_FALLBACK = {
@@ -60,6 +67,16 @@ const STRATEGY_GUIDANCE_FALLBACK = {
avoid: "Avoid when pricing data is missing or outdated.",
example: "Example: Batch or background jobs where lower cost matters most.",
},
"fill-first": {
when: "Use when you want to drain one provider's quota fully before moving to the next.",
avoid: "Avoid when you need request-level load balancing across providers.",
example: "Example: Use all $200 Deepgram credits before falling to Groq.",
},
p2c: {
when: "Use when you want low-latency selection using Power-of-Two-Choices algorithm.",
avoid: "Avoid for small combos with 2 or fewer models — no benefit over round-robin.",
example: "Example: High-throughput inference across 4+ equivalent model endpoints.",
},
};
const ADVANCED_FIELD_HELP_FALLBACK = {
@@ -126,6 +143,25 @@ const STRATEGY_RECOMMENDATIONS_FALLBACK = {
"Use for batch/background jobs where cost is the main KPI.",
],
},
"fill-first": {
title: "Quota drain strategy",
description: "Exhausts one provider's quota before moving to the next in chain.",
tips: [
"Order models by free quota size — biggest first.",
"Enable health checks to skip drained providers.",
"Ideal for free-tier stacking (Deepgram → Groq → NIM).",
],
},
p2c: {
title: "Power-of-Two-Choices",
description:
"Picks the less-loaded of two random candidates per request — low latency at scale.",
tips: [
"Use with 4+ models for best effect.",
"Requires latency telemetry enabled in Settings.",
"Great replacement for round-robin in high-throughput combos.",
],
},
};
const COMBO_USAGE_GUIDE_STORAGE_KEY = "omniroute:combos:hide-usage-guide";
@@ -227,6 +263,8 @@ function getStrategyBadgeClass(strategy) {
if (strategy === "random") return "bg-purple-500/15 text-purple-600 dark:text-purple-400";
if (strategy === "least-used") return "bg-cyan-500/15 text-cyan-600 dark:text-cyan-400";
if (strategy === "cost-optimized") return "bg-teal-500/15 text-teal-600 dark:text-teal-400";
if (strategy === "fill-first") return "bg-orange-500/15 text-orange-600 dark:text-orange-400";
if (strategy === "p2c") return "bg-indigo-500/15 text-indigo-600 dark:text-indigo-400";
return "bg-blue-500/15 text-blue-600 dark:text-blue-400";
}
@@ -1365,10 +1403,24 @@ function ComboFormModal({ isOpen, combo, onClose, onSave, activeProviders }) {
);
};
const FREE_STACK_PRESET_MODELS = [
{ model: "gc/gemini-3-flash-preview", weight: 0 },
{ model: "kr/claude-sonnet-4.5", weight: 0 },
{ model: "if/kimi-k2-thinking", weight: 0 },
{ model: "if/qwen3-coder-plus", weight: 0 },
{ model: "qw/qwen3-coder-plus", weight: 0 },
{ model: "nvidia/llama-3.3-70b-instruct", weight: 0 },
{ model: "groq/llama-3.3-70b-versatile", weight: 0 },
];
const applyTemplate = (template) => {
setStrategy(template.strategy);
setConfig((prev) => ({ ...prev, ...template.config }));
if (!name.trim()) setName(template.suggestedName);
// Pre-fill Free Stack with 7 real free provider models
if (template.id === "free-stack") {
setModels(FREE_STACK_PRESET_MODELS);
}
};
// Format model display name with readable provider name
@@ -1473,7 +1525,12 @@ function ComboFormModal({ isOpen, combo, onClose, onSave, activeProviders }) {
return (
<>
<Modal isOpen={isOpen} onClose={onClose} title={isEdit ? t("editCombo") : t("createCombo")}>
<Modal
isOpen={isOpen}
onClose={onClose}
title={isEdit ? t("editCombo") : t("createCombo")}
size="full"
>
<div className="flex flex-col gap-3">
{/* Name */}
<div>
@@ -2411,6 +2411,7 @@ function ConnectionRow({
<button
onClick={onEdit}
className="p-2 hover:bg-black/5 dark:hover:bg-white/5 rounded text-text-muted hover:text-primary"
title={t("edit")}
>
<span className="material-symbols-outlined text-[18px]">edit</span>
</button>
@@ -2421,7 +2422,11 @@ function ConnectionRow({
>
<span className="material-symbols-outlined text-[18px]">vpn_lock</span>
</button>
<button onClick={onDelete} className="p-2 hover:bg-red-500/10 rounded text-red-500">
<button
onClick={onDelete}
className="p-2 hover:bg-red-500/10 rounded text-red-500"
title={t("deleteConnection")}
>
<span className="material-symbols-outlined text-[18px]">delete</span>
</button>
</div>
@@ -2644,6 +2649,8 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }) {
const [validating, setValidating] = useState(false);
const [validationResult, setValidationResult] = useState(null);
const [saving, setSaving] = useState(false);
const [extraApiKeys, setExtraApiKeys] = useState<string[]>([]);
const [newExtraKey, setNewExtraKey] = useState("");
useEffect(() => {
if (connection) {
@@ -2653,6 +2660,10 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }) {
apiKey: "",
healthCheckInterval: connection.healthCheckInterval ?? 60,
});
// Load existing extra keys from providerSpecificData
const existing = connection.providerSpecificData?.extraApiKeys;
setExtraApiKeys(Array.isArray(existing) ? existing : []);
setNewExtraKey("");
setTestResult(null);
setValidationResult(null);
}
@@ -2739,6 +2750,13 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }) {
updates.rateLimitedUntil = null;
}
}
// Persist extra API keys in providerSpecificData
if (!isOAuth) {
updates.providerSpecificData = {
...(connection.providerSpecificData || {}),
extraApiKeys: extraApiKeys.filter((k) => k.trim().length > 0),
};
}
await onSave(updates);
} finally {
setSaving(false);
@@ -2823,6 +2841,68 @@ function EditConnectionModal({ isOpen, connection, onSave, onClose }) {
</>
)}
{/* T07: Extra API Keys for round-robin rotation */}
{!isOAuth && (
<div className="flex flex-col gap-2">
<label className="text-sm font-medium text-text-main">
Extra API Keys
<span className="ml-2 text-[11px] font-normal text-text-muted">
(round-robin rotation optional)
</span>
</label>
{extraApiKeys.length > 0 && (
<div className="flex flex-col gap-1.5">
{extraApiKeys.map((key, idx) => (
<div key={idx} className="flex items-center gap-2">
<span className="flex-1 font-mono text-xs bg-sidebar/50 px-3 py-2 rounded border border-border text-text-muted truncate">
{`Key #${idx + 2}: ${key.slice(0, 6)}...${key.slice(-4)}`}
</span>
<button
onClick={() => setExtraApiKeys(extraApiKeys.filter((_, i) => i !== idx))}
className="p-1.5 rounded hover:bg-red-500/10 text-red-400 hover:text-red-500"
title="Remove this key"
>
<span className="material-symbols-outlined text-[16px]">close</span>
</button>
</div>
))}
</div>
)}
<div className="flex gap-2">
<input
type="password"
value={newExtraKey}
onChange={(e) => setNewExtraKey(e.target.value)}
placeholder="Add another API key..."
className="flex-1 text-sm bg-sidebar/50 border border-border rounded px-3 py-2 text-text-main placeholder:text-text-muted focus:ring-1 focus:ring-primary outline-none"
onKeyDown={(e) => {
if (e.key === "Enter" && newExtraKey.trim()) {
setExtraApiKeys([...extraApiKeys, newExtraKey.trim()]);
setNewExtraKey("");
}
}}
/>
<button
onClick={() => {
if (newExtraKey.trim()) {
setExtraApiKeys([...extraApiKeys, newExtraKey.trim()]);
setNewExtraKey("");
}
}}
disabled={!newExtraKey.trim()}
className="px-3 py-2 rounded bg-primary/10 text-primary hover:bg-primary/20 disabled:opacity-40 text-sm font-medium"
>
Add
</button>
</div>
{extraApiKeys.length > 0 && (
<p className="text-[11px] text-text-muted">
{extraApiKeys.length + 1} keys total rotating round-robin on each request.
</p>
)}
</div>
)}
{/* Test Connection */}
{!isCompatible && (
<div className="flex items-center gap-3">
+17 -19
View File
@@ -8,38 +8,36 @@
* comes back online.
*/
import { useState, useEffect, useCallback } from "react";
import { useState, useEffect } from "react";
export default function MaintenanceBanner() {
const [show, setShow] = useState(false);
const [message, setMessage] = useState("");
const checkHealth = useCallback(async () => {
try {
const res = await fetch("/api/monitoring/health", {
signal: AbortSignal.timeout(3000),
});
if (res.ok) {
// Server is healthy — hide banner if shown
if (show) {
useEffect(() => {
const checkHealth = async () => {
try {
const res = await fetch("/api/monitoring/health", {
signal: AbortSignal.timeout(3000),
});
if (res.ok) {
setShow(false);
setMessage("");
} else {
setShow(true);
setMessage("Server is experiencing issues. Some features may be unavailable.");
}
} else {
} catch {
setShow(true);
setMessage("Server is experiencing issues. Some features may be unavailable.");
setMessage("Server is unreachable. Reconnecting...");
}
} catch {
setShow(true);
setMessage("Server is unreachable. Reconnecting...");
}
}, [show]);
};
useEffect(() => {
// Check health every 10 seconds
// Run immediately on mount, then every 10 seconds
checkHealth();
const interval = setInterval(checkHealth, 10000);
return () => clearInterval(interval);
}, [checkHealth]);
}, []); // empty deps — checkHealth is defined inside effect, no stale closure
if (!show) return null;