Skip to content

fork: preserve URLs in uv.lock across re-locks (v0.11.14)#2

Closed
harupy wants to merge 23 commits into
mainfrom
fork/preserve-lockfile-urls-v0.11.14
Closed

fork: preserve URLs in uv.lock across re-locks (v0.11.14)#2
harupy wants to merge 23 commits into
mainfrom
fork/preserve-lockfile-urls-v0.11.14

Conversation

@harupy
Copy link
Copy Markdown
Member

@harupy harupy commented May 18, 2026

Summary

Rewrite proxy registry URLs in uv.lock to their canonical counterparts via the UV_INDEX_PROXIES environment variable. Rebased on top of upstream 0.11.14.

Context: astral-sh#6349. When UV_DEFAULT_INDEX points at an internal mirror, upstream uv lock rewrites every source.registry URL in uv.lock to the mirror, creating noisy diffs and breaking portability across environments using different mirrors.

How it works

Set UV_INDEX_PROXIES with canonical:proxy mappings:

export UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple

After resolution, Lock::rewrite_proxy_urls() replaces every matching proxy registry URL with its canonical counterpart in the lockfile. The canonical URLs are also injected into the satisfies() check so that subsequent uv lock runs recognize the lockfile as up-to-date (no unnecessary re-resolve).

Changes vs. prior PR (#1)

  • Rebased onto upstream 0.11.14.
  • Renamed UV_PYPI_PROXIES → UV_INDEX_PROXIES.
  • Apply proxy mappings to the default index used by build resolution (IndexUrls::default_index()).
  • Resolve canonical URLs back to the proxy for uv sync so installs hit the mirror.
  • Also rewrite proxy URLs on Unchanged lock results (no-resolve fast path).
  • Reduce proxy URL log noise — trace! per-package, debug! for summary.
  • Added debug logging for proxy URL resolution.

All fork changes are tagged with // fork: comments so rebasers can find them quickly.

Test plan

  • cargo test -p uv-resolver --lib url_preservation passes locally.
  • Second uv lock run reports Existing uv.lock satisfies workspace requirements (no re-resolve).
  • Fork-release workflow produces per-PR binaries.

🤖 Generated with Claude Code

harupy and others added 23 commits May 18, 2026 14:26
@harupy @claude
fork: preserve URLs in uv.lock across re-locks
7bfab85 
When UV_DEFAULT_INDEX differs across environments (e.g. an internal
mirror on developer machines vs. CI, or across regions), upstream
`uv lock` rewrites `source.registry`, `sdist.url`, and `wheels[].url`
in `uv.lock` to the current mirror's URLs, producing noisy diffs and
breaking lockfile portability (astral-sh#6349).

This fork adds `Lock::rewrite_urls_from`, invoked after `do_lock` in
`LockOperation::execute`, which copies URL fields from the previous
lockfile onto the newly-resolved lock whenever a package is still
present at the same (name, version). Hashes and versions continue
to be written as resolved — only URL fields are held stable.

Non-registry sources (git, direct URL, path, directory, editable,
virtual) are left untouched. Version bumps naturally pick up fresh
URLs because the (name, version) key no longer matches.

Also adds a `Fork release` GitHub Actions workflow that publishes
a rolling `fork-latest` release with linux x86_64 and macOS aarch64
binaries on every push to main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: trigger per-PR, tag releases pr-<number>
9a09fc9 
Workflow now triggers on pull_request (opened/synchronize/reopened)
and publishes a release tagged `pr-<number>` per PR, so each standing
fork PR gets its own binaries without overwriting other PRs' assets.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: use pr-<number> as release title
a129013 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: force-move tag in place instead of delete+recreate
e81972f 
Previously each run deleted the existing release (and its tag) before
recreating it, which briefly left no release present. Now the tag ref
is force-updated via the GitHub API, release notes are edited in
place, and assets are replaced with `gh release upload --clobber`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: SHA-pin actions and add concurrency group
d8fdd1f 
Pin all actions to the same SHAs upstream uv uses:
- actions/checkout v6.0.2
- Swatinem/rust-cache v2.9.1
- actions/upload-artifact v7.0.1
- actions/download-artifact v8.0.1

Add a per-PR concurrency group so a new push to a PR cancels any
in-progress run for the same PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: skip uv.lock write when URL preservation makes it a no-op
8edf734 
When the only difference between the resolver's new lock and the
previous on-disk lock is the index URL (and derived file URLs),
`rewrite_urls_from` produces a new lock equal to the previous one.
Before this change we still wrote that byte-identical content back
to disk, touching the file's mtime and bothering build tools that
watch for modifications.

Now we compare the post-rewrite lock against `previous` and skip
`target.commit(...)` when they match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: rewrite PackageId in every Dependency too, not just Package.id
a2489ae 
`PackageId` implements Hash+Eq over its `source` field. Mutating
`Package.id.source` without touching `Dependency.package_id.source`
left the same logical package keyed under two different hashes,
causing a panic in `installable.rs:508` when `inverse[&package.id]`
could not find the entry that had been inserted under the
dependency's hash.

Fix by applying the preserved-registry URL substitution to every
`PackageId` occurrence in the lock — `Package.id` plus every
`Dependency.package_id` inside `dependencies`,
`optional_dependencies`, and `dependency_groups`.

Added a two-package dependency-edge test that verifies all
`Dependency.package_id.source` values are rewritten and that the
re-serialized lock is byte-identical to the previous one.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: apply rustfmt to url_preservation.rs
219f296 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: rebuild Lock.by_id after URL rewrite; clean up install tmp dir
af32c90 
`Lock.by_id` is a cached `FxHashMap<PackageId, usize>` built once in
`Lock::new`. After `rewrite_urls_from` mutates `Package.id.source`,
the cached keys become stale and `find_by_id` panics at
`lock/mod.rs:1464` with "locked package for ID" during sync/graph
construction (for example via `installable.rs:2223`).

Rebuild `by_id` at the end of `rewrite_urls_from` to reflect the
post-mutation PackageIds. Add a regression test that runs
`find_by_id` on every dependency after a rewrite.

Also update the release-notes install snippets to use `mktemp -d` +
`trap` cleanup so the downloaded tarball and extracted binaries are
removed after installation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: skip remote-index URL check in satisfies() for URL preservation
ae673e7 
URL preservation keeps the previous registry URL in the lockfile, which
won't match the current UV_DEFAULT_INDEX when mirrors differ. The
satisfies() check was returning MissingRemoteIndex on every run,
forcing a full re-resolve even when nothing changed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: normalize newly added packages to canonical registry URL
413bd1f 
When `uv add` introduces a new package, it gets the current
UV_DEFAULT_INDEX mirror URL as its registry source. Previously,
rewrite_urls_from only preserved URLs for packages matching by
(name, version) — new packages kept the mirror URL.

Now infer the canonical registry URL from the previous lockfile and
apply it to all registry packages, including newly added ones, so
the lockfile stays consistent regardless of which mirror resolved
the package.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: replace previous-lock URL inference with UV_PYPI_PROXIES env var
33f8af8 
Replace the complex "infer canonical URL from previous lockfile" logic
with a simple env var mapping. UV_PYPI_PROXIES maps canonical URLs to
proxy URLs (format: canonical:proxy,canonical2:proxy2).

After resolution, rewrite_proxy_urls() replaces every matching proxy
registry URL with its canonical counterpart. This handles both existing
and newly added packages uniformly — no need to track the previous
lockfile at all.

Example:
  UV_PYPI_PROXIES=https://pypi.org/simple:https://pypi-proxy.dev.databricks.com/simple

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: simplify mod.rs — restore upstream code, inject canonical URLs
e48fa2a 
Instead of commenting out the entire remotes collection and check in
satisfies(), restore the original upstream code and just add canonical
URLs from UV_PYPI_PROXIES to the remotes set. This reduces the mod.rs
diff from ~60 lines of comments to 8 added lines.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: replace internal URL with example.com in docs
34eccb6 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: fix UrlString construction — use SmallString::from for &str
2254345 
UrlString doesn't implement From<&str>; use UrlString::new(SmallString::from(...))
to construct from string slices.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: add UV_PYPI_PROXIES setup instructions to release notes
4f78fca 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: rename UV_PYPI_PROXIES to UV_INDEX_PROXIES
fb0a97e 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: also rewrite proxy URLs on Unchanged lock results
f1654e6 
When satisfies() returns Satisfied (lockfile already up-to-date), the
result is Unchanged and the rewrite_proxy_urls code path was skipped.
Now handle both Changed and Unchanged cases so proxy URLs get rewritten
to canonical even when no re-resolution was needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: resolve canonical URLs back to proxy for uv sync
5672067 
The lockfile stores canonical URLs (e.g. pypi.org/simple), but uv sync
needs to fetch from the proxy that is actually reachable. Add proxy_url()
reverse mapping and apply it at the 3 points where RegistrySource::Url
is converted to IndexUrl for HTTP fetching.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: add debug logging for proxy URL resolution
25f1257 
Log when rewriting proxy→canonical (lock) and canonical→proxy (sync)
so the mapping is visible in verbose mode (`uv -v`).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: apply UV_INDEX_PROXIES to default index for build resolution
8448d0b 
When no explicit default index is configured, uv falls back to
pypi.org/simple. If UV_INDEX_PROXIES maps pypi.org to a proxy, the
default index now resolves through the proxy so that build-system.requires
(e.g. setuptools) can be fetched without direct pypi.org access.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: reduce proxy URL log noise — use trace for per-package, debug f…
bb83b9a 
…or summary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: apply UV_INDEX_PROXIES to IndexUrls::default_index() too
ffe4a3f 
The previous fix only patched IndexLocations::default_index(), but the
resolver uses IndexUrls::default_index() which had a separate fallback
to the hardcoded PyPI URL. Apply the same proxy mapping here so that
uv lock also fetches through the proxy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy harupy closed this May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@harupy