38 Commits

Author SHA1 Message Date
Chunyue Wang 6aa4515798 fix(context-engine): gracefully degrade to legacy engine on third-party plugin resolution failure (#66930)
Merged via squash.

Prepared head SHA: 969c67716c717cfde745696db0225110fcfe2d68
Co-authored-by: openperf <80630709+openperf@users.noreply.github.com>
Co-authored-by: openperf <80630709+openperf@users.noreply.github.com>
Reviewed-by: @openperf
2026-04-15 14:59:29 +08:00
Josh Lehman 75e7fc97f8 fix: preserve runtime token budget in deferred context-engine maintenance (#66820)
* fix(context-engine): pass deferred maintenance token budget

Thread tokenBudget through the after-turn runtime context so background context-engine maintenance reuses the real model context window instead of falling back to 128k. Also pass through a best-effort currentTokenCount from the latest call total and make the runtime context type explicit about both fields.

Regeneration-Prompt: |
  OpenClaw already passed the real context token budget into direct context-engine calls like afterTurn and assemble, but deferred maintain() reused only the runtimeContext object and that object did not carry tokenBudget. Lossless Claw therefore fell back to 128k during background maintenance, which made budget-trigger fire much more aggressively than the live model context warranted. Thread the real contextTokenBudget into buildAfterTurnRuntimeContext so deferred maintenance receives the same budget, and pass a straightforward best-effort currentTokenCount from the latest call total while the relevant data is already in scope. Keep the change additive, update the runtime-context type, and cover the background maintenance/runtime-context behavior with focused tests.

* fix(context-engine): use prompt usage for deferred maintenance
2026-04-14 15:30:37 -07:00
fuller-stack-dev 2677f7cf14 fix: validate resolved context engine contracts (#63222)
Merged via squash.

Prepared head SHA: 5f3a15c670ad27898cb83944e485ae002fd9ee49
Co-authored-by: fuller-stack-dev <263060202+fuller-stack-dev@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-13 07:39:34 -07:00
EVA c15b295a85 Run context-engine turn maintenance as idle-aware background work (#65233)
Merged via squash.

Prepared head SHA: e9f6c679ba8709a1be32a18b6963862d2c6a5243
Co-authored-by: 100yenadmin <239388517+100yenadmin@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-04-13 06:50:22 -07:00
Peter Steinberger 87fa88ac3d fix: use literal runtime import for compaction 2026-04-12 18:56:27 +01:00
Vincent Koc f1b08eea54 fix(cycles): cut madge back-edges in agent and cron types 2026-04-12 12:40:25 +01:00
Vincent Koc 74f31241ed fix(runtime): hide lazy command and context seams from static graph 2026-04-12 12:23:19 +01:00
Peter Steinberger e26edee39e fix(context-engine): bundle legacy runtime registration 2026-04-12 00:22:32 +01:00
Nimrod Gutman f04e045815 fix(context-engine): restore bundled legacy engine loading (#64936) 2026-04-11 20:31:49 +03:00
Vincent Koc 747b26ea0f fix(context-engine): lazy-load legacy engine registration 2026-04-11 15:45:19 +01:00
Vincent Koc 74e7b8d47b fix(cycles): bulk extract leaf type surfaces 2026-04-11 13:26:50 +01:00
Vincent Koc 95bc417944 fix(cycles): split residual shared type seams 2026-04-11 02:46:40 +01:00
Peter Steinberger 01113566fd chore: enable await-thenable 2026-04-10 20:14:49 +01:00
Josh Lehman e46e32b98c feat: expose prompt-cache runtime context to context engines (#62179)
* Context engine: plumb prompt cache runtime context

Add a typed prompt-cache payload to the context-engine runtime context and populate it from the embedded runner's resolved retention, last-call usage, cache-break observation, and cache-touch metadata. Also pass the same payload through the retry compaction runtime context when a run attempt already has it.

Regeneration-Prompt: |
  Expose OpenClaw prompt-cache telemetry to context engines in a narrow,
  additive way without changing compaction policy. Keep the public change on
  the OpenClaw side only: add a typed promptCache payload to the context-engine
  runtime context, thread it into afterTurn, and also into compact where the
  existing run loop already has the data cheaply available.

  Use OpenClaw's resolved cache retention, not raw config. Use last-call usage
  for the new payload, not accumulated retry or tool-loop totals. Reuse the
  existing prompt-cache observability result and tracked change causes instead
  of inventing a new heuristic. If cache-touch metadata is already available
  from the cache-TTL bookkeeping, include it; do not invent expiry timestamps
  for providers where OpenClaw cannot know them confidently.

  Keep the interface backward-compatible for engines that ignore the new field.
  Add focused tests around the existing attempt/context-engine helpers and the
  compaction runtime-context propagation path rather than broad new integration
  coverage.

* Agents: fix prompt-cache afterTurn usage

Regeneration-Prompt: |
  Fix PR #62179 so context-engine prompt-cache metadata uses only the current attempt's usage. The review comment pointed out that early exits could reuse a prior turn's assistant usage when no new assistant message was produced. Restrict the prompt-cache lastCallUsage lookup to assistant messages added after prePromptMessageCount, and fall back to current-attempt usage totals instead of stale snapshot history. Also repair the PR's new context-engine test typings and add a regression test for the stale prior-turn case. Two import-only fixes in doctor-state-integrity and config/talk were already broken on origin/main, but they blocked build/check and the gateway-watch regression harness, so include the minimum unblocking imports as well.

* Agents: document prompt-cache context

* Agents: address prompt-cache review feedback

* Doctor: drop unused isRecord import
2026-04-07 09:29:57 -07:00
Vincent Koc 2988203a5e feat(context-engine): add memory prompt helper 2026-04-07 08:56:41 +01:00
Peter Steinberger 27d4992eef Tests: mock context-engine compact runtime seam 2026-04-07 12:29:33 +08:00
Peter Steinberger ee63fdb056 test(ci): harden context engine runtime bridge test 2026-04-03 21:12:56 +01:00
Tak Hoffman f5643544c2 Reduce lint suppressions in core tests and runtime 2026-03-27 02:11:26 -05:00
Peter Steinberger 2fc017788c test: reduce remaining clone seams 2026-03-26 20:01:01 +00:00
Peter Steinberger 9428b38452 refactor: consolidate core runtime state helpers 2026-03-22 18:09:45 +00:00
Peter Steinberger 4229ffe2b9 perf: reduce runtime and test startup overhead 2026-03-21 20:18:16 +00:00
Danh Doan e78129a4d9 feat(context-engine): pass incoming prompt to assemble (#50848)
Merged via squash.

Prepared head SHA: 282dc9264d4157c78959c626bbe6f33ea364def5
Co-authored-by: danhdoan <12591333+danhdoan@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-20 17:03:21 -07:00
Josh Lehman 751d5b7849 feat: add context engine transcript maintenance (#51191)
Merged via squash.

Prepared head SHA: b42a3c28b4395bd8a253c7728080f09100d02f42
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-20 16:28:27 -07:00
John Scianna 5607da90d5 feat: pass modelId to context engine assemble() (#47437)
Merged via squash.

Prepared head SHA: d708ddb222abda2c8d5396bbf4ce9ee5c4549fe3
Co-authored-by: jscianna <9017016+jscianna@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-20 08:05:02 -07:00
Josh Lehman 7f0f8dd268 feat: expose context-engine compaction delegate helper (#49061)
* ContextEngine: add runtime compaction delegate helper

* plugin-sdk: expose compaction delegate through compat

* docs: clarify delegated plugin compaction

* docs: use scoped compaction delegate import
2026-03-17 22:54:18 -07:00
Peter Steinberger 8a9dee9ac8 test: trim redundant context engine assertions 2026-03-17 16:21:58 +00:00
F_ool 094a0cc412 fix(context-engine): preserve legacy plugin sessionKey interop (#44779)
Merged via squash.

Prepared head SHA: e04c6fb47d1ad2623121c907b2e8dcaff62b9ad7
Co-authored-by: hhhhao28 <112874572+hhhhao28@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-17 09:14:14 -07:00
Vincent Koc 7931f06c00 Plugins: harden context engine ownership 2026-03-15 13:51:15 -07:00
Vincent Koc b795ba1d02 Merge branch 'main' of https://github.com/openclaw/openclaw
* 'main' of https://github.com/openclaw/openclaw:
  Plugins: reserve context engine ownership (#47595)
  fix(release): block oversized npm packs that regress low-memory startup (#46850)
  Scripts: rebuild on extension and tsdown config changes (#47571)
  Docs: move release runbook to maintainer repo (#47532)
  docs(zalo): document current Marketplace bot behavior (openclaw#47552)
2026-03-15 13:42:21 -07:00
Vincent Koc 85dd0ab2f8 Plugins: reserve context engine ownership (#47595)
* Plugins: reserve context engine ownership

* Update src/context-engine/registry.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-03-15 13:33:37 -07:00
Vincent Koc 51631e5797 Plugins: reserve context engine ownership 2026-03-15 12:27:29 -07:00
Josh Lehman 50cc375c11 feat(context-engine): plumb sessionKey into all ContextEngine methods (#44157)
Merged via squash.

Prepared head SHA: 0b341f6f4ce487055d8bc0c0d335c42577941592
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-12 12:43:36 -07:00
rabsef-bicrym ff47876e61 fix: carry observed overflow token counts into compaction (#40357)
Merged via squash.

Prepared head SHA: b99eed4329bda45083cdedc2386c2c4041c034be
Co-authored-by: rabsef-bicrym <52549148+rabsef-bicrym@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-12 06:58:42 -07:00
Daniel Reis fbf5d56366 test(context-engine): add bundle chunk isolation tests for registry (#40460)
Merged via squash.

Prepared head SHA: 44622abfbc83120912060abb1059cbca8a20be83
Co-authored-by: dsantoreis <220753637+dsantoreis@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-09 08:15:35 -07:00
Josh Lehman 4bfa800cc7 fix: share context engine registry across bundled chunks (#40115)
Merged via squash.

Prepared head SHA: 6af4820b7d0ea64d96f2f894ef3b0e5750b776aa
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Co-authored-by: jalehman <550978+jalehman@users.noreply.github.com>
Reviewed-by: @jalehman
2026-03-08 11:56:01 -07:00
Peter Steinberger 11be305609 refactor: neutralize context engine runtime bridge 2026-03-08 18:40:15 +00:00
Vincent Koc 063b9aabe2 fix: xxxxx 2026-03-06 19:46:38 -05:00
Josh Lehman fee91fefce feature(context): extend plugin system to support custom context management (#22201)
* feat(context-engine): add ContextEngine interface and registry

Introduce the pluggable ContextEngine abstraction that allows external
plugins to register custom context management strategies.

- ContextEngine interface with lifecycle methods: bootstrap, ingest,
  ingestBatch, afterTurn, assemble, compact, prepareSubagentSpawn,
  onSubagentEnded, dispose
- Module-level singleton registry with registerContextEngine() and
  resolveContextEngine() (config-driven slot selection)
- LegacyContextEngine: pass-through implementation wrapping existing
  compaction behavior for 100% backward compatibility
- ensureContextEnginesInitialized() guard for safe one-time registration
- 19 tests covering contract, registry, resolution, and legacy parity

* feat(plugins): add context-engine slot and registerContextEngine API

Wire the ContextEngine abstraction into the plugin system so external
plugins can register context engines via the standard plugin API.

- Add 'context-engine' to PluginKind union type
- Add 'contextEngine' slot to PluginSlotsConfig (default: 'legacy')
- Wire registerContextEngine() through OpenClawPluginApi
- Export ContextEngine types from plugin-sdk for external consumers
- Restore proper slot-based resolution in registry

* feat(context-engine): wire ContextEngine into agent run lifecycle

Integrate the ContextEngine abstraction into the core agent run path:

- Resolve context engine once per run (reused across retries)
- Bootstrap: hydrate canonical store from session file on first run
- Assemble: route context assembly through pluggable engine
- Auto-compaction guard: disable built-in auto-compaction when
  the engine declares ownsCompaction (prevents double-compaction)
- AfterTurn: post-turn lifecycle hook for ingest + background
  compaction decisions
- Overflow compaction: route through contextEngine.compact()
- Dispose: clean up engine resources in finally block
- Notify context engine on subagent lifecycle events

Legacy engine: all lifecycle methods are pass-through/no-op, preserving
100% backward compatibility for users without a context engine plugin.

* feat(plugins): add scoped subagent methods and gateway request scope

Expose runtime.subagent.{run, waitForRun, getSession, deleteSession}
so external plugins can spawn sub-agent sessions without raw gateway
dispatch access.

Uses AsyncLocalStorage request-scope bridge to dispatch internally via
handleGatewayRequest with a synthetic operator client. Methods are only
available during gateway request handling.

- Symbol.for-backed global singleton for cross-module-reload safety
- Fallback gateway context for non-WS dispatch paths (Telegram/WhatsApp)
- Set gateway request scope for all handlers, not just plugin handlers
- 3 staleness tests for fallback context hardening

* feat(context-engine): route /compact and sessions.get through context engine

Wire the /compact command and sessions.get handler through the pluggable
ContextEngine interface.

- Thread tokenBudget and force parameters to context engine compact
- Route /compact through contextEngine.compact() when registered
- Wire sessions.get as runtime alias for plugin subagent dispatch
- Add .pebbles/ to .gitignore

* style: format with oxfmt 0.33.0

Fix duplicate import (ControlUiRootState in server.impl.ts) and
import ordering across all changed files.

* fix: update extension test mocks for context-engine types

Add missing subagent property to bluebubbles PluginRuntime mock.
Add missing registerContextEngine to lobster OpenClawPluginApi mock.

* fix(subagents): keep deferred delete cleanup retryable

* style: format run attempt for CI

* fix(rebase): remove duplicate embedded-run imports

* test: add missing gateway context mock export

* fix: pass resolved auth profile into afterTurn compaction

Ensure the embedded runner forwards resolved auth profile context into
legacy context-engine compaction params on the normal afterTurn path,
matching overflow compaction behavior. This allows downstream LCM
summarization to use the intended provider auth/profile consistently.

Also fix strict TS typing in external-link token dedupe and align an
attempt unit test reasoningLevel value with the current ReasoningLevel
enum.

Regeneration-Prompt: |
  We were debugging context-engine compaction where downstream summary
  calls were missing the right auth/profile context in normal afterTurn
  flow, while overflow compaction already propagated it. Preserve current
  behavior and keep changes additive: thread the resolved authProfileId
  through run -> attempt -> legacy compaction param builder without
  broad refactors.

  Add tests that prove the auth profile is included in afterTurn legacy
  params and that overflow compaction still passes it through run
  attempts. Keep existing APIs stable, and only adjust small type issues
  needed for strict compilation.

* fix: remove duplicate imports from rebase

* feat: add context-engine system prompt additions

* fix(rebase): dedupe attempt import declarations

* test: fix fetch mock typing in ollama autodiscovery

* fix(test): add registerContextEngine to diffs extension mock APIs

* test(windows): use path.delimiter in ios-team-id fixture PATH

* test(cron): add model formatting and precedence edge case tests

Covers:
- Provider/model string splitting (whitespace, nested paths, empty segments)
- Provider normalization (casing, aliases like bedrock→amazon-bedrock)
- Anthropic model alias normalization (opus-4.5→claude-opus-4-5)
- Precedence: job payload > session override > config default
- Sequential runs with different providers (CI flake regression pattern)
- forceNew session preserving stored model overrides
- Whitespace/empty model string edge cases
- Config model as string vs object format

* test(cron): fix model formatting test config types

* test(phone-control): add registerContextEngine to mock API

* fix: re-export ChannelKind from config-reload-plan

* fix: add subagent mock to plugin-runtime-mock test util

* docs: add changelog fragment for context engine PR #22201
2026-03-06 05:31:59 -08:00