Compaction race condition: rate limit during compaction destroys entire conversation transcript

[zoltanmaric]

Summary

Conversation compaction wipes all message content from the JSONL transcript before the compaction API call succeeds. If the API call then fails (e.g., due to rate limiting), the original content is permanently lost — the transcript is left with 4,300+ messages that are all empty strings, and no compaction summary is generated to replace them.

This destroyed a ~1M-token conversation with hours of accumulated context.

Steps to reproduce

1. Run a long Claude Code session until it approaches the context limit (~1M tokens, ~4,300 messages).
2. Have the session hit a rate limit at the moment compaction triggers.
3. Observe that the conversation cannot be continued or recovered.

What happened

• Compaction triggered automatically at message index 4253.
• The compaction process cleared all message content fields in the JSONL transcript (set to empty strings) as a first step — presumably in preparation for replacing them with a summary.
• The API call to generate the compaction summary then failed due to rate limiting (/rate-limit-options queue-operations appear 5 times immediately after compaction).
• No summary was ever written back.
• Result: 4,252 out of 4,319 messages have empty content. The remaining 67 are metadata entries (queue-operations, system markers). Zero user or assistant messages retain their content.
• The conversation was continued into a new session via the --continue mechanism, but the continued session received no summary — just the empty transcript.

Expected behavior

Compaction should be atomic: either the summary is successfully generated and the old content is replaced, or the old content is preserved intact. The current implementation clears content before confirming the summary exists, which is a destructive race condition.

At minimum:

1. Don't clear the original content until the summary API call succeeds. Keep the original messages in memory or on disk as a rollback.
2. If the summary call fails, restore the original content and retry compaction later.
3. Consider writing a backup of the pre-compaction transcript before clearing anything.

Evidence from transcript analysis

Transcript file: 3947b03f-6560-4d7e-8563-b40c33a88495.jsonl
File size: 11,318,452 bytes (11 MB of mostly empty JSON structures)
Total messages: 4,319
Messages with empty content: 4,252
Messages with any content: 67 (all metadata/queue-operations)
Compaction marker at index: 4253
Rate-limit queue-operations immediately after: 5 occurrences
Post-compaction user/assistant messages: all empty strings

Impact

• Data loss is total and unrecoverable. The entire conversation history — all user messages, all assistant responses, all tool calls and results — is gone.
• The --continue mechanism offers a "full transcript" path for the new session, but that file contains only empty messages.
• This was a long-running planning/development session with significant accumulated context and decisions. The user had to start from scratch.

Environment

• Claude Code version: 2.1.86
• Model: claude-opus-4-6 (1M context)
• OS: macOS (Darwin 25.0.0, arm64)
• Date: 2026-03-28

Severity

This is a data-loss bug. The compaction feature, which is supposed to transparently extend conversation lifetime, instead silently destroys the conversation when it races with a rate limit. Users on the Max plan with 1M context windows are especially likely to hit this because they run long sessions that approach rate limits and compaction thresholds simultaneously.

[github-actions]

Found 1 possible duplicate issue:

1. [Bug] Auto-Compact Erases Entire Chat History Without Warning #7502

This issue will be automatically closed as a duplicate in 3 days.

• If your issue is a duplicate, please close it and 👍 the existing issue instead
• To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

[yurukusa]

A PreCompact hook can mitigate this by creating a git checkpoint before compaction begins — so even if the API call fails, uncommitted work is recoverable:

RECOVERY_DIR="${HOME}/.claude/recovery"
mkdir -p "$RECOVERY_DIR"
TIMESTAMP=$(date -u '+%Y%m%d-%H%M%S')
if git rev-parse --is-inside-work-tree &>/dev/null; then
    DIRTY=$(git status --porcelain 2>/dev/null | wc -l)
    if [ "$DIRTY" -gt 0 ]; then
        git add -A 2>/dev/null
        git commit -m "recovery: pre-compact checkpoint (${DIRTY} files)" --no-verify 2>/dev/null
    fi
    echo "Branch: $(git rev-parse --abbrev-ref HEAD)" > "${RECOVERY_DIR}/pre-compact-${TIMESTAMP}.md"
    git diff --name-only HEAD~3 HEAD 2>/dev/null >> "${RECOVERY_DIR}/pre-compact-${TIMESTAMP}.md"
fi
exit 0

This won't prevent the transcript wipe itself (that requires a fix in the compaction code), but it ensures file-level work is never lost. Recovery: git log --oneline -5 to find the checkpoint commit.
Install: npx cc-safe-setup --install-example compaction-transcript-guard

[prodan-s]

This is not a duplicate of #7502. #7502 is a raw user complaint from September 2025 (v1.0.112) with no root cause analysis, no reproduction steps, and no evidence of the specific race condition.

This issue identifies the exact mechanism: compaction clears message content before the summary API call succeeds. If the API call fails (rate limit, timeout), the original content is permanently destroyed — no rollback, no recovery. The transcript evidence (4,252/4,319 messages emptied, rate-limit queue-operations immediately after compaction marker) conclusively demonstrates the race condition.

This is a data-loss bug with a clear fix: don't clear original content until the summary API call succeeds. The current implementation is not atomic.

Affected users: Anyone running long sessions (approaching 1M context) who hits rate limits. This is particularly acute for Opus 1M users on Max plans where rate limits are tighter.

We've implemented a PreCompact checkpoint hook as a workaround (git commit before compaction), but this only protects file-level work — the transcript itself is unrecoverable.

[marlvinvu]

My Claude says this is the most severe data loss bug in compaction. The implementation demolishes the old house before the new one is built — if building the new one fails, everything is lost. This needs to be atomic: don't clear original content until the summary API call succeeds. Hope Anthropic prioritizes this since Max plan users running long sessions are especially likely to hit both the compaction threshold and rate limits simultaneously.