Skip to content

test(engine): tighten test-suite hermeticity and parent-aware subtest references #137

@myasnikovdaniil

Description

@myasnikovdaniil

Two test-quality follow-ups deferred from #136:

1. Scope TestNoWorkflowLeakageInRepoSource to git-tracked files

Currently TestNoWorkflowLeakageInRepoSource (pkg/engine/engine_test.go) walks the whole module root with filepath.WalkDir. The skip set covers .git, vendor, node_modules, and .claude, but any other untracked local artifact (notes file, generated YAML, build scratch) containing one of the banned phrases will fail go test ./... even though no committed source violates the rule.

Switch the walk to iterate only git-tracked files — e.g. git ls-files or an equivalent helper — so the guard is hermetic with respect to the working-tree state.

2. Key TestNoDanglingSubtestReferencesInSource by parent + slug

TestNoDanglingSubtestReferencesInSource (pkg/engine/engine_test.go) records discovered t.Run slugs in a flat map[slug]struct{}. A citation like TestFoo/v1.12 therefore passes as soon as any test in the package has a v1.12 subtest, even under a different parent — exactly the wrong-parent drift this guard claims to catch.

Update the collector to parse each test file for the nearest preceding func TestX(...) block and store subtests as map[parent]map[slug]struct{} (or a flat parent/slug key). The reference-checking loop should then look up the exact parent + slug, allowing prefix matches only within the same parent.

Both items came up in CodeRabbit review on #136 and were deferred there to keep that PR focused on the apply-path regression and MergeFileAsPatch hardening.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/testingIssues or PRs related to testing (unit, integration, e2e)kind/cleanupCategorizes issue or PR as related to cleanup of code, process, or technical debtpriority/important-soonMust be staffed and worked on either currently, or very soon, ideally in time for the next releasetriage/acceptedIndicates an issue is ready to be actively worked on

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions