test(integration): unstick 23 stale tests that drift from production code#1457
Merged
Conversation
…code Integration Tests have been silently skipped in CI/CD Pipeline since the early-stage Build & Test jobs were failing on Windows; once Windows is green (#1456), the integration matrix runs for the first time in weeks and surfaces accumulated test rot: - test_git_cache_{hermetic,phase3w5}: GitCache now lays out checkouts under <shard>/<sha>/<variant> (perf #1433); tests still expected the pre-variant <shard>/<sha> path. Add the 'full' variant segment and pass-through kwargs (partial, sparse_paths, promisor_url) to mock assertions. - test_context_optimizer_{phase3w4,placement}: ContextOptimizer fallback path reads instruction.file_path.stem; MagicMock(spec= Instruction) does not expose dataclass fields, so set file_path on the helper. - test_mcp_integrator_{install_flow,phase3w4}::test_registry_overlay _warns_when_string: _apply_overlay no longer warns on string 'registry' (silent no-op). Invert the assertion. - test_mcp_integrator_{characterisation,phase3w5}::test_skips_parse _errors_and_continues: assertion depended on Path.iterdir order (not guaranteed across OSes). Match on the set of called paths and key the ValueError side-effect by path identity. - test_wave8_codex_download_coverage::test_remote_only_rejected: Codex now accepts streamable-http remote-only servers; only SSE remotes are rejected. Set transport_type='sse' so the test actually exercises the rejection branch. Full integration suite: 9000 passed, 222 skipped, 2 xfailed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR repairs accumulated integration-test drift so the newly re-enabled integration matrix runs cleanly again (especially on Windows), aligning tests with recent production behavior changes in GitCache, MCP integration, ContextOptimizer, and the Codex adapter.
Changes:
- Update GitCache integration tests to reflect the new checkout layout under
<shard>/<sha>/<variant>/(including thefull/variant) and updated call signatures. - Adjust MCP integrator tests to match current overlay behavior (no warning for string
registry) and make parse-error characterization resilient to filesystem enumeration order. - Refresh Codex and ContextOptimizer integration tests to match new remote-only transport rules and the Instruction fallback behavior (
instruction.file_path.stem).
Show a summary per file
| File | Description |
|---|---|
| tests/integration/test_wave8_codex_download_coverage.py | Updates Codex remote-only test to exercise the SSE rejection branch via transport_type. |
| tests/integration/test_mcp_integrator_phase3w5.py | Makes parse-error test resilient to nondeterministic apm.yml discovery ordering. |
| tests/integration/test_mcp_integrator_phase3w4.py | Updates overlay warning expectation for string registry to the current silent-noop behavior. |
| tests/integration/test_mcp_integrator_install_flow.py | Same overlay warning expectation update as phase3w4, but in install-flow coverage. |
| tests/integration/test_mcp_integrator_characterisation.py | Same parse-error ordering resilience update as phase3w5, in characterization coverage. |
| tests/integration/test_git_cache_phase3w5.py | Updates GitCache tests for variant-aware checkout paths and new ensure/create call shapes. |
| tests/integration/test_git_cache_hermetic.py | Same GitCache test updates as phase3w5, in the hermetic suite. |
| tests/integration/test_context_optimizer_placement.py | Ensures mocked Instruction includes file_path to match production fallback behavior. |
| tests/integration/test_context_optimizer_phase3w4.py | Same Instruction mock fix as placement suite. |
Copilot's findings
- Files reviewed: 9/9 changed files
- Comments generated: 4
| warnings.simplefilter("always") | ||
| MCPIntegrator._apply_overlay(cache, dep) | ||
| assert any("registry" in str(w.message) for w in caught) | ||
| assert not any("registry" in str(w.message) for w in caught) |
| warnings.simplefilter("always") | ||
| MCPIntegrator._apply_overlay(cache, dep) | ||
| assert any("registry" in str(w.message) for w in caught) | ||
| assert not any("registry" in str(w.message) for w in caught) |
Comment on lines
+305
to
+309
| def _from_apm_yml(path, *_args, **_kw): | ||
| if path == first: | ||
| raise ValueError("bad") | ||
| return pkg | ||
|
|
Comment on lines
+305
to
+309
| def _from_apm_yml(path, *_args, **_kw): | ||
| if path == first: | ||
| raise ValueError("bad") | ||
| return pkg | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Integration Tests have been silently skipped in CI/CD Pipeline because Build & Test was failing on Windows. Once #1456 made Windows green, the integration matrix ran for the first time in weeks and surfaced accumulated test rot — 23 stale tests across 9 files.
Fixes
test_git_cache_{hermetic,phase3w5}(14)<shard>/<sha>/<variant>(perf #1433). Tests expected pre-variant layout.test_context_optimizer_{phase3w4,placement}(6)instruction.file_path.stem;MagicMock(spec=Instruction)does not expose dataclass fields.test_mcp_integrator_{install_flow,phase3w4}::test_registry_overlay_warns_when_string(2)_apply_overlayno longer warns on stringregistry(silent no-op).test_mcp_integrator_{characterisation,phase3w5}::test_skips_parse_errors_and_continues(2)Path.iterdirordering (not guaranteed across filesystems).test_wave8_codex_download_coverage::test_remote_only_rejected(1)Validation
Full local integration run:
9000 passed, 222 skipped, 2 xfailed in 311.62s. All lint gates pass.Why this matters
This unblocks the v0.14.2 release pipeline (run 26311614440).