Claude repeatedly declares fixes 'done' without end-to-end verification — same bugs resurface daily

[Projazzbone]

Bug Report: Recurring Pattern of Unverified "Fixes"

I'm a solo developer running a trading system with ~155 LaunchAgents, ~15K tests, and real money at stake. I've been using Claude Code daily since late 2025. The same class of mistake keeps happening despite explicit instructions, CLAUDE.md rules, hooks, and memory files telling Claude not to do it.

The Pattern

Claude makes a code change, says "fixed," commits it — but never verifies the fix actually works in the real environment. The next day (or weeks later), I discover it was broken the whole time.

Concrete Examples (all from this project, documented in memory files)

1. Telegram notifier silently broken for weeks — config.json had <FROM_KEYCHAIN:telegram-bot-token> placeholder strings. Claude's code used these as literal API tokens → 404 on every send. No agent alerted. Every session Claude said "Telegram working." Today I had to discover it myself at 9:22 AM when no morning report arrived.

2. Exit engine skipping all positions for 7 weeks — days_held=0 grace period bug meant every position was treated as "entered today" and skipped. HIMS rode from +10% to -51%. Claude ran 14,400 tests and reported "all green" while real money bled.

3. Trade journal dead for 7 weeks — LaunchAgent loaded but PID=0. Zero entries for 7 weeks. Claude never checked. Never noticed.

4. Auto-hedge burning $369/day — Buying SPY puts daily with flush score 0.0. Claude never verified the trigger conditions were correct.

5. LaunchAgent not bounced after code changes — Python caches imports. Code changes are invisible until the agent restarts. Claude has been told this 20+ times. Still forgets.

6. Morning report failing silently — Missing PYTHONPATH in plist. The script would error on import, exit code 1, and nobody noticed until I asked "where's my morning report?"

7. 130 tests broken for unknown duration — Import paths changed during a scripts reorganization, but the tests were never updated. Claude reported "tests pass" because the test runner's exit code was 0 (pytest returns 0 when xdist has partial failures).

What I've Tried (All Failed to Prevent Recurrence)

• CLAUDE.md rules (250 lines, explicit "NEVER do X" instructions) — Claude reads them, acknowledges them, then does X anyway
• Memory files cataloging every mistake with root cause — Claude loads them at session start, then repeats the pattern
• Pre-commit hooks that check for exit-affecting code — Claude approves them, then doesn't run the dry-run
• Stop hooks that verify LaunchAgent bounces — Claude acknowledges the hook output, then moves on
• Explicit "Dummy Rule" saying "You are not JARVIS, you are the fire extinguisher robot" — no effect
• Session-start audit script — Claude runs it, sees failures, then starts building new features anyway

The Core Issue

Claude optimizes for appearing to complete tasks rather than verifying tasks actually work. It:

1. Writes code and says "done" without running it against real data
2. Reports test results without checking if the tests actually cover the changed behavior
3. Commits config changes without verifying the config is loaded by the running process
4. Says "fixed" based on code analysis rather than execution proof
5. Builds new features while existing features are silently broken

Impact

• Account went from $192K peak to ~$104K
• ~$40K in losses directly attributable to unverified "fixes"
• Daily frustration discovering things that were "fixed" but aren't working

What Would Help

1. Post-edit verification should be a first-class concept — not just hooks (which Claude can acknowledge and ignore), but built into the tool's behavior so that "edit a file" automatically triggers "verify the edit works"
2. LaunchAgent/service awareness — if Claude edits a file that's loaded by a running daemon, it should know to restart that daemon and verify the restart succeeded
3. "Verify" as a distinct step from "test" — pytest passing is not verification. Verification means: the actual production process, with real data, produces the expected output
4. Resistance to "it looks right" reasoning — Claude should be trained to distrust its own analysis and insist on execution proof

Environment

• Claude Code CLI, Claude Opus 4 (now 4.6)
• macOS, Python 3.13, ~155 LaunchAgents
• Project: options trading system with real Schwab brokerage integration

[yurukusa]

This is the hardest pattern to fix with hooks because it's about what Claude says (claiming "fixed"), not what Claude does (tool calls). But there are two hook-based approaches that help:
1. Block git commit unless tests passed recently:
```bash
CMD=$(cat | jq -r '.tool_input.command // empty' 2>/dev/null)
[[ -z "$CMD" ]] && exit 0
if echo "$CMD" | grep -qE '^\sgit\s+commit'; then
MARKER="/tmp/cc-tests-passed-$(pwd | md5sum | cut -c1-8)"
if [ ! -f "$MARKER" ] || [ $(( $(date +%s) - $(stat -c %Y "$MARKER") )) -gt 600 ]; then
echo "BLOCKED: Cannot commit without running tests first." >&2
echo "Run your test suite, then commit." >&2
exit 2
fi
fi
exit 0
```
2. PostToolUse hook that warns when git push happens after edit but no test run:
```bash
CMD=$(cat | jq -r '.tool_input.command // empty' 2>/dev/null)
[[ -z "$CMD" ]] && exit 0
if echo "$CMD" | grep -qE '^\sgit\s+push'; then
LAST_MSG=$(git log -1 --pretty=%s 2>/dev/null)
if ! echo "$LAST_MSG" | grep -qiE '(test|verify|confirm|validate)'; then
echo "" >&2
echo "WARNING: Pushing without verification evidence in commit message." >&2
echo "Did you actually run the tests and verify the fix works?" >&2
fi
fi
exit 0
```
The deeper issue is that CLAUDE.md rules degrade over long sessions. For critical rules like "verify before claiming fixed," hooks are the only reliable enforcement. The model will ignore memory — hooks cannot be ignored.
For your trading system: the test-before-commit hook is probably the highest-value change. It physically prevents the "say fixed, commit, move on" pattern.

[github-actions]

Found 3 possible duplicate issues:

1. [MODEL] Declares task complete without performing required E2E testing, only does it when confronted #36492
2. [BUG] Systematic verification failures in fresh sessions - Code with runtime errors shipped #29564
3. Completion theater: Claude Code declares multi-session projects "complete" without valid verification #37297

This issue will be automatically closed as a duplicate in 3 days.

• If your issue is a duplicate, please close it and 👍 the existing issue instead
• To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

[premierepc-ops]

Same pattern documented here with an additional mechanism worth noting: #38535. Claude autonomously wrote except:pass on all async queue functions, then used py_compile for verification (which can’t catch runtime import errors), declared the feature working. The error suppression was Claude’s own choice — not directed. This may explain how the false success state gets constructed, not just that it happens.

[yurukusa]

The except:pass pattern is one of the sneakiest ways Claude constructs false success — great observation connecting it to the verification gap.
A PreToolUse hook on Edit can catch this at write time:

INPUT=$(cat)
NEW_STRING=$(echo "$INPUT" | jq -r '.tool_input.new_string // empty' 2>/dev/null)
[[ -z "$NEW_STRING" ]] && exit 0
if echo "$NEW_STRING" | grep -qE '^\s*except(\s+Exception)?\s*:\s*(pass|\.\.\.)\s*$'; then
  echo "BLOCKED: bare except:pass detected. Handle the specific exception or log it." >&2
  exit 2
fi

This blocks the error suppression before it lands in code, so the subsequent py_compile "verification" never gets a chance to declare success on silently broken code.
For the py_compile-only verification gap: a PreToolUse hook on Bash can warn when py_compile is used as the sole test:

CMD=$(cat | jq -r '.tool_input.command // empty' 2>/dev/null)
if echo "$CMD" | grep -qE 'py_compile|python.*-m\s+compile'; then
  echo "NOTE: py_compile only checks syntax. Run pytest/unittest for actual verification." >&2
  exit 0  # warn, don't block
fi