feat(compaction): opt-in hard-floor for incremental compaction below contextThreshold

[100yenadmin]

TL;DR

Adds an opt-in config flag respectThresholdAsHardFloor (default false, no behavior change for existing users). When enabled, evaluateIncrementalCompaction short-circuits with reason="below-context-threshold-floor" whenever currentTokenCount < contextThreshold * tokenBudget, regardless of cache state, leaf-trigger fires, or activity-band heuristics. The deferred-mode debt-recording path at engine.ts:6548 is gated under the same predicate to prevent silent leakage of the floor through the deferred drain.

This is a policy enhancement, not a bug fix. The default behavior continues to allow cold-cache catch-up passes to fire below threshold (the existing design that some users want). Users who instead want strict "never compact below X%" semantics — typically because of bursty/idle conversation patterns where step-away gaps cause cold-mode firing to bleed context — can now opt in with one flag.

The problem this addresses, visualized

Without the flag (existing behavior):
   ┌─────────────────────────────────────────────────────────────────┐
   │  step-away 20 min  →  cache cold  →  next turn               │
   │      ↓                                                        │
   │  cold-mode catch-up fires (maxColdCacheCatchupPasses leaf passes) │
   │      ↓                                                        │
   │  context drops by ~2× chunkSize even though %-used was low    │
   │      ↓                                                        │
   │  step-away 5 more min  →  cache cold again  →  fires again    │
   │      ↓                                                        │
   │  pattern repeats; context drains during idle gaps             │
   └─────────────────────────────────────────────────────────────────┘

With respectThresholdAsHardFloor=true:
   ┌─────────────────────────────────────────────────────────────────┐
   │  evaluateIncrementalCompaction first checks:                  │
   │    currentTokenCount  vs  floor (= contextThreshold × budget) │
   │      │                                                        │
   │      ├── below floor → return shouldCompact=false             │
   │      │                  reason="below-context-threshold-floor"│
   │      │                  (cache-aware/leaf-trigger gates skipped)│
   │      │                                                        │
   │      └── at/above floor → existing logic runs unchanged       │
   │                            (budget-trigger, hot-cache gates,  │
   │                             cold catch-up, etc.)              │
   └─────────────────────────────────────────────────────────────────┘

Behavior matrix

Flag	currentTokenCount	Decision
false (default)	any	existing logic — all gates and triggers behave as before
true	undefined	falls through to existing logic (no floor enforced when count is unknown)
true	< contextThreshold × tokenBudget	short-circuit: shouldCompact=false reason="below-context-threshold-floor"
true	≥ contextThreshold × tokenBudget	falls through to existing logic
true (deferred mode)	< floor	also skips recordDeferredCompactionDebt (prevents drain leak)
true (deferred mode)	≥ floor	existing deferred-debt recording behavior

Diff overview

3 commits:

• 1c99131 feat(compaction): add opt-in respectThresholdAsHardFloor flag

  • src/db/config.ts: add respectThresholdAsHardFloor: boolean to LcmConfig, parser entry with env-var support (LCM_RESPECT_THRESHOLD_AS_HARD_FLOOR), default false.
  • src/engine.ts: 30-line guard at the top of evaluateIncrementalCompaction, returns through the existing logIncrementalCompactionDecision so the new reason shows up in standard telemetry.
  • Tests in test/engine.test.ts and test/config.test.ts.

• 434eb28 fix(compaction): plug deferred-mode hard-floor leak; add boundary tests

  • In deferred mode (engine.ts:6548-6603), debt is recorded based on thresholdDecision.shouldCompact || rawLeafTrigger?.shouldCompact — both use unguarded compaction.evaluate / evaluateLeafTrigger. Without this fix, debt would accumulate from below-floor leaf-trigger fires and later drain via paths that don't re-check the floor.
  • Adds the same predicate guard around recordDeferredCompactionDebt and the deferred-drain scheduling, with an explanatory log line: [lcm] afterTurn: skipping deferred compaction debt below context-threshold floor ….
  • Adds boundary tests: exact equality at floor, undefined currentTokenCount with flag enabled, and an integration test for the deferred-mode skip.

• 8cc4e63 feat(manifest): expose respectThresholdAsHardFloor in plugin config schema and uiHints

  • openclaw.plugin.json: adds respectThresholdAsHardFloor: { type: "boolean" } to configSchema.properties (required so OpenClaw's host-side validator with additionalProperties: false accepts the new field). Adds a uiHints entry with label and help text so the field shows up in the OpenClaw plugin settings UI.

Naming conventions

• Flag: respectThresholdAsHardFloor — follows the existing pattern of compound-clause boolean flags.
• Env var: LCM_RESPECT_THRESHOLD_AS_HARD_FLOOR — matches the existing LCM_* env pattern for plugin config overrides.
• Reason string: "below-context-threshold-floor" — follows the hyphenated convention of "budget-trigger", "hot-cache-defer", "hot-cache-budget-headroom", "below-leaf-trigger".
• Telemetry: routed through the existing logIncrementalCompactionDecision so the new reason appears in the same [lcm] incremental compaction decision: … lines operators already grep.

What is intentionally NOT changed

• compactFullSweep and compactUntilUnder are not affected. Forced compaction (manualCompaction: true, overflow recovery, explicit compact({force: true})) bypasses evaluateIncrementalCompaction entirely and continues to work above the soft floor as before. This matters for emergency overflow recovery, which should always be allowed.
• Default behavior is preserved exactly — when the flag is false (the default), zero lines of pre-existing logic execute differently. Verified via the regression test evaluateIncrementalCompaction preserves default behavior when respectThresholdAsHardFloor is disabled.

Tests

• test/engine.test.ts — 5 new tests:

  • evaluateIncrementalCompaction short-circuits below contextThreshold when respectThresholdAsHardFloor is enabled — verifies short-circuit and that evaluateLeafTrigger/evaluate are NOT called.
  • evaluateIncrementalCompaction allows existing compaction logic at/above contextThreshold even when hard-floor is enabled — verifies budget-trigger still fires above floor.
  • evaluateIncrementalCompaction preserves default behavior when respectThresholdAsHardFloor is disabled — regression guard for the default path.
  • evaluateIncrementalCompaction allows existing logic at exact floor boundary even with hard-floor enabled — boundary test for currentTokenCount === minTokenFloor (uses < not <=, so the boundary admits compaction).
  • evaluateIncrementalCompaction falls through to existing logic when currentTokenCount is undefined even with hard-floor enabled — verifies the type-guard fall-through case.
  • afterTurn in deferred mode skips deferred-debt recording when below contextThreshold floor — integration test for the deferred-mode leak fix (commit 434eb28).

• test/config.test.ts — 3 path coverages:

  • Default false in the hardcoded-defaults test.
  • Plugin-config override path.
  • Env-var override path (LCM_RESPECT_THRESHOLD_AS_HARD_FLOOR=true).

Full suite: 860/860 passing.

Independence

PR #619 is independent of PR #621. They address different concerns:

• feat(compaction): opt-in hard-floor for incremental compaction below contextThreshold #619 (this PR): policy enhancement — opt-in hard-floor for incremental compaction.
• fix(afterTurn): evaluate compaction even when ingestBatch is empty #621: bug fix — afterTurn early-return on empty ingestBatch skips compaction evaluation entirely (different code path).

No merge-order dependency. Either PR can land first.

Test plan

• npm run build — bundles cleanly (689kb).
• npm test — 860/860 passing.
• Default-false preserves existing behavior across all 855 pre-existing tests (zero regressions).
• Adversarial review (3 reviewers, separately checking flag interaction, regression risk, and mechanism) — all cleared.
• Manifest schema accepts the new field; verified with the host's config validator (additionalProperties: false).

[100yenadmin]

Adversarial review pass — one HIGH finding fixed

Ran a 3-agent parallel adversarial review on the initial commit. Two reviewers independently flagged a HIGH-severity leak in deferred mode: the hard-floor only short-circuited evaluateIncrementalCompaction, but the after-turn flow at engine.ts:6548 records deferred debt based on thresholdDecision.shouldCompact || rawLeafTrigger?.shouldCompact — both of which use unguarded compaction.evaluate / compaction.evaluateLeafTrigger calls. Below-floor leaf-trigger fires would accumulate debt that later drained via paths that didn't re-check the floor.

Fixed in 434eb28 by gating deferred-debt recording on the same floor predicate. Operators get one info-level log line per skipped recording so suppression is observable.

Other findings + dispositions

Severity	Finding	Disposition
HIGH	Deferred-mode debt leak	Fixed in 434eb28
MEDIUM	New "below-context-threshold-floor" reason isn't recognized by shouldForceDeferredPromptCacheLeafCompaction (engine.ts:2400) or the deferred maintenance.reason === "threshold" check (engine.ts:3196)	Working as intended — both correctly return false / no-match for the new reason since the floor short-circuit is meant to be terminal, not retried
LOW	Boundary tests: exact equality, undefined currentTokenCount with flag enabled	Added (2 new tests in 434eb28)
LOW	activityBand: "low" hardcoded in short-circuit telemetry	Working as intended; documented via the patch comment
✓	Manual / forced compaction (compact({force:true}), compactUntilUnder, compactFullSweep)	Confirmed bypass evaluateIncrementalCompaction and continue to work as intended

Suite

864/864 passing (3 new tests added since the initial commit).