feat(cpp-phase2): PR-03 HTTP loader, Windows/Darwin walker, CI workflow by shivasurya · Pull Request #681 · shivasurya/code-pathfinder

shivasurya · 2026-05-03T23:06:50Z

Summary

PR-03 of the C/C++ Phase 2 stdlib stack. Wires the loaders to the network and stands up the GitHub Actions matrix that produces the six manifest sets (linux/{c,cpp}, windows/{c,cpp}, darwin/{c,cpp}) the loader expects.

Stacked on #680 — must merge AFTER PR-02. Base branch is shiva/cpp-phase2-pr02-loader.

What's in here

HTTP loader (graph/callgraph/registry/clike_http.go, c_stdlib_remote.go, cpp_stdlib_remote.go): fills in PR-02's stubbed loadManifestFromHTTP / fetchHeaderFromHTTP paths. Optional sha256:<hex> checksum verification, 24h disk-cache freshness, stale-cache fallback on network failure so offline scans keep resolving stdlib calls. Backed by httptest coverage (200 / 404 / network failure / checksum match + mismatch / parallel fetches / embedded URL / garbage body).
Generator: Windows + Darwin paths in tools/internal/clikeextract/walker_xplat.go. Probes /usr/x86_64-w64-mingw32 (mingw-w64 on Ubuntu), /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk and the Apple libc++ tree. All probe paths are package vars so tests don't need a real toolchain.
GitHub Actions (.github/workflows/generate-clike-stdlib.yml): three-runner matrix (Linux, Linux+mingw, macos-latest) → six artifacts → R2 upload + CDN verification. R2 steps gated on secret presence so forks and pre-secret runs still pass.

Local-first, then remote

By design, the default --stdlib-base-url stays empty in this PR. Operators need to:

Confirm the workflow generates all six manifests on a one-off workflow_dispatch run (no secrets needed).
Provision R2_ACCOUNT_ID / R2_ACCESS_KEY_ID / R2_SECRET_ACCESS_KEY and verify the CDN serves the manifests.
Switch the default URL to https://assets.codepathfinder.dev/registries in a follow-up so users get cross-platform resolution out of the box.

Out of scope (deferred)

Default URL switch to the CDN — see "Local-first" above
Platform-specific YAML overlay entries (Win32 specifics, dispatch.h, mach.h, CoreFoundation C bindings) — best refined against actual workflow output
Live large-codebase validation (proxygen, sglang) — manual operator step

Verification

gradle buildGo — clean
go test ./... — all packages pass
golangci-lint run ./... — 0 issues
Coverage: 91.7% registry, 91.8% clikeextract; new HTTP code 75-100% per function

Test plan

CI green
Manual workflow_dispatch of generate-clike-stdlib.yml produces all 6 manifest.json files (R2 upload skipped until secrets are wired)
After R2 setup: end-to-end scan with --stdlib-base-url=https://assets.codepathfinder.dev/registries matches PR-02's local resolution numbers

🤖 Generated with Claude Code

Fills in PR-02's stubbed loadManifestFromHTTP / fetchHeaderFromHTTP paths so the remote loaders work end-to-end against a CDN. Local file:// behavior is unchanged. clike_http.go shared helpers: - fetchURL: single GET, 200-only, error message embeds the URL so CDN-side issues are diagnosable without bouncing through retries. - verifyChecksum: optional sha256:<hex> verification; empty checksum disables verification (manifests pre-PR-03 ship without it). - joinURL: trailing/leading slash-tolerant URL composition shared between C and C++. C and C++ HTTP loaders now: - Construct paths as <baseURL>/<platform>/<lang>/v1/<file>, prefer per-entry URL when the manifest embeds one. - Try the on-disk cache first when fresh (24h TTL); always fall back to stale cache on network failure so offline scans still resolve stdlib calls. - Best-effort write to the on-disk cache after a successful fetch; cache write failures warn but never block the in-memory result. Updated tests: - New httptest-driven coverage for both loaders: 200, 404, network failure with cache fallback, checksum match, checksum mismatch, parallel fetches, embedded URL, garbage body. - Two PR-02 stub assertions (HTTPMode_Stub, HTTPMode_FetchHeaderStub) rewritten to point at an unreachable port and assert the proper network error. - cmd/scan_stdlib_test.go now isolates XDG_CACHE_HOME / HOME when exercising the unreachable-host path so a developer cache cannot produce a false-positive success. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Replaces the PR-01 PR-03-error stubs in DiscoverHeaderSources with working source-tree probes for the four cross-platform combinations the workflow targets next. walker_xplat.go: - windowsCSource / windowsCppSource probe the mingw-w64 install at /usr/x86_64-w64-mingw32 (the layout `apt install mingw-w64` produces on ubuntu-latest). The C source covers Win32 + MSVCRT; the C++ source picks the freshest libstdc++ version directory. - darwinCSource / darwinCppSource probe Apple's Command Line Tools first and fall back to a full Xcode install. The C source uses the macOS SDK; the C++ source uses the clang-shipped libc++ tree. - findVersionedDir, firstExistingDir, detectMingwVersion, detectDarwinSDKTag, detectDarwinCppTag are small helpers that derive the version tag stamped into manifest.SystemTag. All probe paths are package vars rather than literals so tests can override them to exercise both hit and miss branches without depending on whether the host actually has mingw/Xcode installed. Two PR-01 tests rewritten to assert the new "headers not found" error messages instead of the deferred-PR-03 stub. Coverage on the new helpers: 75-100%. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

Three-runner matrix that produces the six manifest sets the loader expects (linux/{c,cpp}, windows/{c,cpp}, darwin/{c,cpp}): - generate-linux ubuntu-latest with libc6-dev + libstdc++-13-dev - generate-windows ubuntu-latest with mingw-w64 (Win32 + libstdc++) - generate-darwin macos-latest using Command Line Tools' SDK + libc++ tree The publish job collects the artifacts, asserts all 6 manifest.json files are present, and uploads to Cloudflare R2 under the same bucket/prefix scheme the existing Go stdlib pipeline uses (s3://code-pathfinder-assets/registries/<platform>/<lang>/v1/). The R2 upload + CDN verification steps run only when both R2_ACCOUNT_ID and R2_ACCESS_KEY_ID are configured — keeps the workflow green on forks and during operator setup. Triggered on push to main when generator code or overlays change, plus manual workflow_dispatch. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

safedep · 2026-05-03T23:06:54Z

SafeDep Report Summary

No dependency changes detected. Nothing to scan.

View complete scan results →

_{This report is generated by SafeDep Github App}

github-actions · 2026-05-03T23:07:20Z

Code Pathfinder Security Scan

No security issues detected.

Metric	Value
Files Scanned	13
Rules	205

_{Powered by Code Pathfinder}

codecov · 2026-05-03T23:09:07Z

Codecov Report

❌ Patch coverage is 89.95816% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.67%. Comparing base (6204dda) to head (2d5c149).

Files with missing lines	Patch %	Lines
...gine/graph/callgraph/registry/cpp_stdlib_remote.go	79.59%	6 Missing and 4 partials ⚠️
...engine/graph/callgraph/registry/c_stdlib_remote.go	87.23%	3 Missing and 3 partials ⚠️
...engine/tools/internal/clikeextract/walker_xplat.go	93.10%	3 Missing and 3 partials ⚠️
sast-engine/graph/callgraph/registry/clike_http.go	94.44%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@                       Coverage Diff                        @@
##           shiva/cpp-phase2-pr02-loader     #681      +/-   ##
================================================================
+ Coverage                         85.61%   85.67%   +0.05%     
================================================================
  Files                               200      202       +2     
  Lines                             28912    29141     +229     
================================================================
+ Hits                              24754    24966     +212     
- Misses                             3195     3203       +8     
- Partials                            963      972       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

shivasurya and others added 3 commits May 3, 2026 19:05

shivasurya added enhancement New feature or request github_actions Pull requests that update GitHub Actions code go Pull requests that update go code labels May 3, 2026

shivasurya self-assigned this May 3, 2026

shivasurya added enhancement New feature or request github_actions Pull requests that update GitHub Actions code go Pull requests that update go code labels May 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(cpp-phase2): PR-03 HTTP loader, Windows/Darwin walker, CI workflow#681

feat(cpp-phase2): PR-03 HTTP loader, Windows/Darwin walker, CI workflow#681
shivasurya wants to merge 3 commits intoshiva/cpp-phase2-pr02-loaderfrom
shiva/cpp-phase2-pr03-remote

shivasurya commented May 3, 2026

Uh oh!

safedep Bot commented May 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 3, 2026

Uh oh!

codecov Bot commented May 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

shivasurya commented May 3, 2026

Summary

What's in here

Local-first, then remote

Out of scope (deferred)

Verification

Test plan

Uh oh!

safedep Bot commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SafeDep Report Summary

Uh oh!

github-actions Bot commented May 3, 2026

Code Pathfinder Security Scan

Uh oh!

codecov Bot commented May 3, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

safedep Bot commented May 3, 2026 •

edited

Loading