Skip to content

SEP-023: track_test_failure.sh blind to wrapped test invocations #13

@samudzi

Description

@samudzi

Summary

Test commands piped through error-catching wrappers (e.g., npm test 2>&1 | tee test.log) exit 0 even on test failure, so PostToolUseFailure never fires and tests_failed is never set. The TDD gate remains locked.

Motivation

This creates a subtle failure mode where the model wraps test commands for logging purposes and inadvertently bypasses the TDD gate. The test appears to fail (output shows failures) but the gate doesn't unlock because the exit code was 0.

Proposed Change

Instruct Claude in CLAUDE.md to run test commands bare, not wrapped. Add explicit guidance:

  • Do not pipe test commands through tee, 2>&1, or other wrappers
  • Run test commands directly so exit codes are preserved
  • If logging is needed, use set -o pipefail before piped commands

Acceptance Criteria

  • CLAUDE.md includes explicit instruction to run tests bare
  • Consider adding pipefail detection in track_test_failure.sh
  • Model correctly triggers TDD gate when tests fail

Metadata

Metadata

Assignees

No one assigned

    Labels

    SEPSoftware Evolution ProposalbugSomething isn't workingminorMinor issues

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions