Add unvendored repository support with precise dependency diff analysis#71
Open
bts-bastion wants to merge 8 commits into
Open
Add unvendored repository support with precise dependency diff analysis#71bts-bastion wants to merge 8 commits into
bts-bastion wants to merge 8 commits into
Conversation
Detect and parse go.work files to resolve packages across all workspace modules, enabling cross-module dependency tracking in CI pipelines. Add -no-workspace flag to opt out. Mark all packages dirty when go.work or go.mod changes. Co-authored by: Blue Thunder Somogyi
These methods enable distinguishing local (main module) packages from external dependencies and mapping changed dependency modules to the local packages that import them. Foundation for unvendored repo support. Co-authored by: Blue Thunder Somogyi
When packages.Load returns packages with errors (common for external dependencies in unvendored repos), skip them when building the forward/reverse dependency graphs. Module info is still recorded. Co-authored by: Blue Thunder Somogyi
When a package in the dependency graph cannot be resolved (e.g., an external dependency in an unvendored repo), skip it instead of returning an error from ChangedPackages(). This prevents GTA from crashing on unvendored repositories. Co-authored by: Blue Thunder Somogyi
When traversing the dependency graph in markedPackages(), skip edges to external (non-local) packages. Uses a type assertion so custom Packager implementations retain the existing traversal behavior. In GOPATH mode (no modules), all packages are treated as local. Co-authored by: Blue Thunder Somogyi
New file gomod.go provides diffGoMod() and diffGoSum() for precise dependency change detection. diffGoMod compares Require and Replace directives using modfile.Parse with ParseLax fallback. diffGoSum does line-level comparison to catch transitive dependency changes. Co-authored by: Blue Thunder Somogyi
BaseFileReader enables reading file content at the git merge-base, used to retrieve old go.mod/go.sum for dependency diff analysis. SetBaseGoMod and SetBaseGoSum options provide the same capability for file-differ mode (no git access). Co-authored by: Blue Thunder Somogyi
When go.mod or go.sum changes in an unvendored repo, GTA now parses the diff to identify exactly which dependency modules changed, then marks only the local packages that import those modules. This replaces the previous "mark all packages" approach which negated GTA's value. Falls back to mark-all when base file content is unavailable (e.g., custom Differ without BaseFileReader, no SetBaseGoMod option). go.work changes continue to use mark-all since workspace structural changes warrant full re-evaluation. Co-authored by: Blue Thunder Somogyi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GTA (Go Test Auto) was designed for vendored repositories. In an unvendored repo, two things break:
Crashes:
packages.Loadreturns packages with errors for external dependencies that aren'tvendored locally. These errors propagate through graph construction and
PackageFromImport,causing GTA to fail outright.
Nuclear go.mod handling: When
go.modchanges, GTA marks every package in the dependencygraph as dirty. In a vendored repo this rarely happens (vendor changes show up as file diffs
instead). In an unvendored repo, routine dependency bumps trigger a full rebuild of the entire
module — completely negating GTA's value as a targeted analysis tool.
Approach
The fix is delivered in two phases across 7 atomic commits, each independently reviewable and
test-green.
Phase 1: Don't crash (commits 1-4)
Four defensive changes make GTA survive unvendored repos without changing its analysis behavior:
Skip errored packages in graph construction (
packager.go): Whenpackages.Loadreturns apackage with
pkg.Errors(e.g., an external dep whose source isn't local), record its module infobut don't add it to the forward/reverse graphs. This prevents broken nodes from polluting
traversal.
Tolerate failures (
gta.go): InChangedPackages(), when the reversegraph contains an external package that can't be resolved,
continueinstead of returning anerror. For vendored repos this path is never hit (all graph packages resolve).
Filter non-local edges during traversal (
gta.go): Thetraverse()function inmarkedPackages()now skips edges to external packages via alocalPackageCheckertype assertion.This prevents the graph walk from chasing into external dependency trees that don't exist locally.
isLocalPackagehelper (packager.go): Determines whether an import path belongs to a mainmodule by checking against
modulesNamesByDir. In GOPATH mode (wheremodulesNamesByDirisempty), all packages are treated as local — preserving existing behavior exactly.
All four changes use type assertions on unexported interfaces, not additions to the public
Packagerinterface. CustomPackagerimplementations are completely unaffected — the typeassertion simply fails and the code falls through to existing behavior.
Phase 2: Precise dependency diff (commits 5-7)
Replace the nuclear "mark all packages when go.mod changes" with targeted analysis:
How
diffGoModworksdiffGoMod(oldData, newData)parses both go.mod files usingmodfile.Parse(withParseLaxfallback for non-standard version strings) and compares:
It returns a
[]ModuleChangeidentifying exactly which dependency module paths changed.How
diffGoSumcatches transitive changesThis is the key insight for correctness.
go.modonly lists direct dependencies. A transitivedependency three levels deep can change version without
go.modchanging at all — butgo.sumwill change, because it records the cryptographic hash of every module in the build graph.
diffGoSum(oldData, newData)does a line-level symmetric diff of the two go.sum files. Eachgo.sum line is
module version hash. Any line present in one file but not the other means thatmodule changed. The function extracts the module path from each differing line and returns the
deduplicated set.
This makes the detection intentionally overprotective: if any transitive dependency changes
its resolved version or hash, go.sum will reflect it, and GTA will mark the local packages that
import that module. It may mark slightly more than strictly necessary (e.g., if a transitive dep
changed but only affects a code path your local package doesn't use), but it will never miss a
change that could affect your build. The bias is toward safety — false positives (extra test runs)
over false negatives (missed regressions).
How the pieces connect at the call site
When
go.modorgo.sumappears in the diff,markedPackages()now:Retrieves old content via
BaseFileReader(the git differ readsgit show <base>:<path>)or via
SetBaseGoMod/SetBaseGoSumoptions (for file-differ mode without git).Diffs both files:
diffGoModfor direct dependency changes,diffGoSumfor transitive.Results are merged and deduplicated.
Maps modules to local importers:
LocalImportersOf(changedModPaths)scans the forwarddependency graph to find which local packages import any package under the changed module paths.
Only those packages are marked dirty.
Falls back to nuclear when base content is unavailable (custom
DifferwithoutBaseFileReader, noSetBaseGoModoption). This preserves existing behavior for any setupthat can't provide the old file content.
What about
go.work?go.workchanges continue to use the nuclear option. Workspace structural changes (adding/removingmodules from the workspace) warrant full re-evaluation since they can fundamentally change which
packages are visible and how they resolve.
Interface compatibility
This PR adds zero methods to any exported interface:
localPackageCheckertraverse()localImporterFindermarkedPackages()BaseFileReaderDifferat call siteBaseFileReaderis a standalone exported interface, not an extension ofDiffer. The git differsatisfies it; the file differ does not. Call sites use
if reader, ok := g.differ.(BaseFileReader).packageContext(the concrete type behindPackager) implements all three. ExternalPackagerimplementations don't need to change — the type assertions simply fail and existing code paths run.
Correctness argument
The detection is sound (no false negatives) because:
go.mod→ caught bydiffGoModgo.sum→ caught bydiffGoSumLocalImportersOfThe detection may produce false positives (overprotective) because:
go.sumline changes don't distinguish which specific packages within a module changeddiffGoSumtreats hash changes the same as version changesThis is the right tradeoff: running a few extra tests is cheap; missing a broken dependency is not.
Test plan
go test -v -count=1 -race ./...— all pass, no racespackager_test.go,gomod_test.go,gta_test.goSetBaseGoModoption_/GOPATHsubtests inTestGTA_ChangedPackages)