Summary
When bktec retries a pytest collection error (e.g. an import failure), it constructs an invalid node ID because the scope is empty. This causes no tests ran on the retry attempt.
The test still correctly fails overall — the collection error is reported as a failure and bktec exits non-zero. But the retry is wasted and the error message is confusing.
Root cause
In internal/runner/pytest.go line 106, bktec constructs retry node IDs by joining scope and name:
Path: fmt.Sprintf("%s::%s", test.Scope, test.Name)For normal test failures this produces valid node IDs like tests/foo.py::test_bar.
For collection errors, the scope is empty — and correctly so. A collection error's node ID is just the file path (e.g. tests/foo.py) with no :: separator. When the test-collector-python plugin splits on ::, it gets scope="" and name="tests/foo.py". The unconditional join produces:
This is not a valid pytest node ID, so pytest reports no tests ran.
Suggested fix
Handle empty scope when constructing the retry path:
if test.Scope != "" {
path = fmt.Sprintf("%s::%s", test.Scope, test.Name)
} else {
path = test.Name
}This would produce tests/foo.py for collection errors, which is a valid pytest node ID. Pytest would attempt to collect the file again — it would still fail (the import error can't be fixed by re-running), but the error message would be clear and the retry behavior would be consistent.
Context
This issue was discovered while investigating buildkite/test-collector-python#106, where collection errors were silently dropped from the JSON report. The fix in buildkite/test-collector-python#108 adds collection errors to the report, which surfaces this pre-existing issue in bktec's retry logic.
Reproduction
https://github.com/altana-ai/bktec-collection-error-repro — the third scenario (03-bktec-retry) demonstrates the invalid node ID on retry after applying the test-collector-python fix.
Summary
When bktec retries a pytest collection error (e.g. an import failure), it constructs an invalid node ID because the scope is empty. This causes
no tests ranon the retry attempt.The test still correctly fails overall — the collection error is reported as a failure and bktec exits non-zero. But the retry is wasted and the error message is confusing.
Root cause
In
internal/runner/pytest.goline 106, bktec constructs retry node IDs by joining scope and name:For normal test failures this produces valid node IDs like
tests/foo.py::test_bar.For collection errors, the scope is empty — and correctly so. A collection error's node ID is just the file path (e.g.
tests/foo.py) with no::separator. When the test-collector-python plugin splits on::, it getsscope=""andname="tests/foo.py". The unconditional join produces:This is not a valid pytest node ID, so pytest reports
no tests ran.Suggested fix
Handle empty scope when constructing the retry path:
This would produce
tests/foo.pyfor collection errors, which is a valid pytest node ID. Pytest would attempt to collect the file again — it would still fail (the import error can't be fixed by re-running), but the error message would be clear and the retry behavior would be consistent.Context
This issue was discovered while investigating buildkite/test-collector-python#106, where collection errors were silently dropped from the JSON report. The fix in buildkite/test-collector-python#108 adds collection errors to the report, which surfaces this pre-existing issue in bktec's retry logic.
Reproduction
https://github.com/altana-ai/bktec-collection-error-repro — the third scenario (
03-bktec-retry) demonstrates the invalid node ID on retry after applying the test-collector-python fix.