Elim dups and improve output by lzehrung · Pull Request #104 · lzehrung/codegraph

lzehrung · 2026-05-23T03:20:28Z

No description provided.

Copilot

Pull request overview

This PR reduces duplicated logic across the codebase (tests, CLI, language definitions, resolution utilities, renderers) and improves output by (a) coalescing duplicate-detector group results and (b) surfacing high-confidence duplicate opportunities in inspect output along with a follow-up duplicates command.

Changes:

Added shared helpers (test temp dirs/path normalization, graph edge keying, package exports target selection, whitespace tokenizer, CSS-like language definition fragments, C-family query fragments, MCP dependency schema, shared JVM import override wrapper).
Enhanced duplicate detection output by coalescing groups with identical primary ranges, adding guardrails for very different-sized grouped units, and filtering groups by minConfidence.
Extended codegraph inspect JSON output with a bounded duplicates section and updated CLI regression tests/docs accordingly.

Reviewed changes

Copilot reviewed 52 out of 52 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/ts-paths-workspace.test.ts	Uses shared test filesystem helpers for tmp dir + path normalization.
tests/symbols-detailed-edgecases.test.ts	Uses shared `mkTmpDir` helper.
tests/sqlite.test.ts	Uses shared tmp dir + path normalization helpers for incremental sqlite assertions.
tests/sql-review-context.test.ts	Switches to shared `mkTmpDir` and passes explicit prefix.
tests/sql-artifact-graph.test.ts	Uses shared `mkTmpDir` for SQL graph fixtures.
tests/robust-fast-graph.test.ts	Uses shared tmp dir + path normalization and shared edge-key helper.
tests/resolution-precedence.test.ts	Uses shared `mkTmpDir` helper.
tests/parsed-cache-reuse.test.ts	Uses shared `mkTmpDir` helper.
tests/parsed-cache-eviction.test.ts	Uses shared `mkTmpDir` helper.
tests/node-modules-and-paths.test.ts	Uses shared tmp dir + path normalization helper.
tests/monorepo-fast-graph.test.ts	Uses shared `graphEdgeKey` helper for parity comparisons.
tests/helpers/graph.ts	New shared graph-test helpers (edge keying + `edgeFrom`).
tests/helpers/filesystem.ts	Adds shared `mkTmpDir` and `normalizeTestPath` for tests.
tests/fast-graph.test.ts	Uses shared `mkTmpDir` + `graphEdgeKey` helper.
tests/fast-graph-edgecases.test.ts	Uses shared tmp dir/path normalization and shared `edgeFrom`.
tests/dynamic-resolution.test.ts	Uses shared tmp dir + path normalization helper.
tests/duplicates.test.ts	Adds coverage for coalescing duplicate groups with identical primary ranges.
tests/disk-cache-sqlite.test.ts	Uses shared `mkTmpDir` helper.
tests/cli-regressions.test.ts	Extends `inspect` regression coverage to assert `duplicates` output + recommended command.
src/util/workspace.ts	Dedupes `package.json#exports` target selection via shared helper.
src/util/resolution/node.ts	Dedupes node_modules `exports` target selection via shared helper.
src/util/projectFiles.ts	Exports `matchesDiscoveryGlob` for reuse by CLI glob filtering.
src/util/packageExports.ts	New shared `exports` target selection helper.
src/sqlite/canned-query.ts	Dedupes direct deps/dependents sqlite query loader via parameterized helper.
src/mcp/tools.ts	Dedupes MCP deps/rdeps input schema via `dependencyInputSchema()`.
src/mcp/server.ts	Dedupes deps/rdeps handler logic via `collectMcpDependencyEntries`.
src/languages/definitions/less.ts	Reuses extracted CSS-like definition fragments.
src/languages/definitions/cssLike.ts	New shared CSS-like structure/graph/nodeTypes helpers.
src/languages/definitions/css.ts	Reuses extracted CSS-like definition fragments.
src/languages/definitions/cpp.ts	Reuses extracted C-family query fragments + split points.
src/languages/definitions/cFamily.ts	Adds shared C-family split points and query-building utilities.
src/languages/definitions/c.ts	Reuses extracted C-family query fragments + split points.
src/indexer/imports/languageSpecific.ts	Dedupes Java/Kotlin import override plumbing via `applyJvmStatementOverride`.
src/impact/suggestions.ts	Reuses shared `toRange` helper from `src/util/ast.ts`.
src/impact/report-suggestions.ts	Dedupes line counting across covered/total coverage sets.
src/graphs/symbol-render.ts	Extracts shared file/symbol render model used by Mermaid + DOT outputs.
src/duplicates.ts	Coalesces groups with same primary ranges; adds nearby-chunk clustering + length-ratio guard; filters groups by confidence threshold.
src/cli/search.ts	Reuses shared `CliAgentCommandContext` type.
src/cli/inspect.ts	Adds bounded `duplicates` section to `inspect` output + recommended `duplicates` command; refactors indexing path.
src/cli/graph.ts	Extracts shared compact symbol projection builder for compact JSON output.
src/cli/explain.ts	Reuses shared `CliAgentCommandContext` type.
src/cli/context.ts	Reuses exported `matchesDiscoveryGlob`; centralizes shared agent command context type.
src/cli/artifact.ts	Reuses shared `CliAgentCommandContext` type.
src/chunking/tokenizer.ts	New shared whitespace token counter.
src/chunking/chunkTextFile.ts	Reuses shared whitespace tokenizer type/impl.
src/chunking/chunkFile.ts	Reuses shared whitespace tokenizer type/impl.
src/agent/handles.ts	Dedupes parsing logic for file-like handles (`file:` / `graph:`).
src/agent-tools.ts	Dedupes dependency tool response mapping and tightens return types.
README.md	Documents `inspect` duplicates output (but example needs refresh—see comment).
docs/superpowers/plans/2026-05-23-eliminate-duplicate-findings.md	Adds a detailed refactor/verification plan and scan notes for duplicates cleanup.
docs/cli.md	Updates CLI docs to mention `inspect` duplicate opportunities + follow-up command.
codegraph-skill/codegraph/SKILL.md	Updates skill guide to mention `inspect` duplicate opportunities.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 52 out of 52 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 52 out of 52 changed files in this pull request and generated 1 comment.

+    const variantsByKey = new Map<string, DuplicateSuggestion>();
+    for (const group of grouped) {
+      for (const variant of group.variants) {
+        variantsByKey.set(suggestionVariantKey(variant), variant);
+      }
+    }
+    const variants = Array.from(variantsByKey.values()).sort(compareSuggestionsForPrimary).slice(0, variantLimit);
+    const rawPairCount = grouped.reduce((count, group) => count + group.rawPairCount, 0);
+    let score = primary.score;
+    let confidence = primary.confidence;
+    let cloneType = primary.cloneType;
+    for (const group of grouped.slice(1)) {
+      score = Math.max(score, group.score);
+      confidence = bestConfidence(confidence, group.confidence);
+      cloneType = bestCloneType(cloneType, group.cloneType);
+    }
+    coalesced.push({
+      ...primary,
+      id: shortHashText(key),
+      score,
+      confidence,
+      cloneType,
+      variants,
+      variantCount: variants.length,
+      rawPairCount,
+      omittedVariantCount: Math.max(0, rawPairCount - variants.length),
+      reasons: mergeGroupReasons(grouped),


lzehrung added 12 commits May 22, 2026 22:00

Plan duplicate elimination work

4588086

Share chunk token and range helpers

44f740b

Share compact symbol projection

9280694

Share symbol render model

6255cb5

Consolidate dependency tool wrappers

89b7a35

Share CSS-like language definition pieces

2dc93db

Share smaller CLI and query helpers

5b8e7ba

Share duplicate test helpers

8811200

Refine duplicate grouping and remaining test helpers

b030468

Mark duplicate refactor plan complete

7f2fcbe

Use static canned query SQL

918c660

Surface duplicate opportunities in inspect

6b55db6

lzehrung changed the title ~~Elim dups~~ Elim dups and improve output May 23, 2026

lzehrung requested a review from Copilot May 23, 2026 03:56

Copilot started reviewing on behalf of lzehrung May 23, 2026 03:56 View session

Copilot AI reviewed May 23, 2026

View reviewed changes

Comment thread tests/helpers/graph.ts

Comment thread README.md

Address PR duplicate cleanup review comments

47a5098

lzehrung requested a review from Copilot May 23, 2026 04:31

Copilot started reviewing on behalf of lzehrung May 23, 2026 04:31 View session

Copilot AI reviewed May 23, 2026

View reviewed changes

Comment thread src/graphs/symbol-render.ts Outdated

Comment thread README.md Outdated

Address follow-up PR review comments

f05a222

lzehrung requested a review from Copilot May 23, 2026 12:29

Copilot started reviewing on behalf of lzehrung May 23, 2026 12:29 View session

Copilot AI reviewed May 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Elim dups and improve output#104

Elim dups and improve output#104
lzehrung wants to merge 14 commits into
mainfrom
elim-dups

lzehrung commented May 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lzehrung commented May 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants