Skip to content

fork: preserve URLs in uv.lock across re-locks#1

Closed
harupy wants to merge 23 commits into
mainfrom
fork/preserve-lockfile-urls
Closed

fork: preserve URLs in uv.lock across re-locks#1
harupy wants to merge 23 commits into
mainfrom
fork/preserve-lockfile-urls

Conversation

@harupy
Copy link
Copy Markdown
Member

@harupy harupy commented Apr 24, 2026 •
edited
Loading

Summary

Rewrite proxy registry URLs in uv.lock to their canonical counterparts via the UV_PYPI_PROXIES environment variable.

Context: astral-sh#6349. When UV_DEFAULT_INDEX points at an internal mirror, upstream uv lock rewrites every source.registry URL in uv.lock to the mirror, creating noisy diffs and breaking portability across environments using different mirrors.

How it works

Set UV_PYPI_PROXIES with canonical:proxy mappings:

export UV_PYPI_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple

After resolution, Lock::rewrite_proxy_urls() replaces every matching proxy registry URL with its canonical counterpart in the lockfile. The canonical URLs are also injected into the satisfies() check so that subsequent uv lock runs recognize the lockfile as up-to-date (no unnecessary re-resolve).

Changes

  • New file: crates/uv-resolver/src/lock/url_preservation.rs — UV_PYPI_PROXIES parsing, Lock::rewrite_proxy_urls(), and canonical_urls() helper.
  • crates/uv-resolver/src/lock/mod.rs — +2 lines (mod url_preservation;) and +6 lines (inject canonical URLs into remotes set in satisfies()).
  • crates/uv/src/commands/project/lock.rs — +10 lines, -4 lines: call rewrite_proxy_urls() after do_lock, skip write if rewrite makes new lock identical to previous.
  • New file: .github/workflows/fork-release.yml — publishes per-PR release binaries (linux x86_64, macOS aarch64).

All fork changes are tagged with // fork: comments so rebasers can find them quickly.

Test plan

  • cargo test -p uv-resolver --lib url_preservation — 4 unit tests:
    • rewrites_proxy_to_canonical — proxy URL rewritten to canonical.
    • rewrites_dependency_package_ids — dependency PackageId sources and by_id cache stay consistent.
    • no_env_var_is_noop — no UV_PYPI_PROXIES → no changes.
    • no_match_leaves_urls_untouched — non-matching proxy → URLs unchanged.
  • Second uv lock run completes in ~7ms with Existing uv.lock satisfies workspace requirements (no re-resolve).
  • Fork-release workflow produces per-PR binaries.

🤖 Generated with Claude Code

harupy and others added 23 commits April 25, 2026 00:43
@harupy @claude
fork: preserve URLs in uv.lock across re-locks
d0ee572 
When UV_DEFAULT_INDEX differs across environments (e.g. an internal
mirror on developer machines vs. CI, or across regions), upstream
`uv lock` rewrites `source.registry`, `sdist.url`, and `wheels[].url`
in `uv.lock` to the current mirror's URLs, producing noisy diffs and
breaking lockfile portability (astral-sh#6349).

This fork adds `Lock::rewrite_urls_from`, invoked after `do_lock` in
`LockOperation::execute`, which copies URL fields from the previous
lockfile onto the newly-resolved lock whenever a package is still
present at the same (name, version). Hashes and versions continue
to be written as resolved — only URL fields are held stable.

Non-registry sources (git, direct URL, path, directory, editable,
virtual) are left untouched. Version bumps naturally pick up fresh
URLs because the (name, version) key no longer matches.

Also adds a `Fork release` GitHub Actions workflow that publishes
a rolling `fork-latest` release with linux x86_64 and macOS aarch64
binaries on every push to main.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: trigger per-PR, tag releases pr-<number>
f22f8f1 
Workflow now triggers on pull_request (opened/synchronize/reopened)
and publishes a release tagged `pr-<number>` per PR, so each standing
fork PR gets its own binaries without overwriting other PRs' assets.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: use pr-<number> as release title
96df320 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: force-move tag in place instead of delete+recreate
42680a1 
Previously each run deleted the existing release (and its tag) before
recreating it, which briefly left no release present. Now the tag ref
is force-updated via the GitHub API, release notes are edited in
place, and assets are replaced with `gh release upload --clobber`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: SHA-pin actions and add concurrency group
32d8e2d 
Pin all actions to the same SHAs upstream uv uses:
- actions/checkout v6.0.2
- Swatinem/rust-cache v2.9.1
- actions/upload-artifact v7.0.1
- actions/download-artifact v8.0.1

Add a per-PR concurrency group so a new push to a PR cancels any
in-progress run for the same PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: skip uv.lock write when URL preservation makes it a no-op
d13bcac 
When the only difference between the resolver's new lock and the
previous on-disk lock is the index URL (and derived file URLs),
`rewrite_urls_from` produces a new lock equal to the previous one.
Before this change we still wrote that byte-identical content back
to disk, touching the file's mtime and bothering build tools that
watch for modifications.

Now we compare the post-rewrite lock against `previous` and skip
`target.commit(...)` when they match.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: rewrite PackageId in every Dependency too, not just Package.id
e242c34 
`PackageId` implements Hash+Eq over its `source` field. Mutating
`Package.id.source` without touching `Dependency.package_id.source`
left the same logical package keyed under two different hashes,
causing a panic in `installable.rs:508` when `inverse[&package.id]`
could not find the entry that had been inserted under the
dependency's hash.

Fix by applying the preserved-registry URL substitution to every
`PackageId` occurrence in the lock — `Package.id` plus every
`Dependency.package_id` inside `dependencies`,
`optional_dependencies`, and `dependency_groups`.

Added a two-package dependency-edge test that verifies all
`Dependency.package_id.source` values are rewritten and that the
re-serialized lock is byte-identical to the previous one.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: apply rustfmt to url_preservation.rs
1a5290e 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: rebuild Lock.by_id after URL rewrite; clean up install tmp dir
6639b6f 
`Lock.by_id` is a cached `FxHashMap<PackageId, usize>` built once in
`Lock::new`. After `rewrite_urls_from` mutates `Package.id.source`,
the cached keys become stale and `find_by_id` panics at
`lock/mod.rs:1464` with "locked package for ID" during sync/graph
construction (for example via `installable.rs:2223`).

Rebuild `by_id` at the end of `rewrite_urls_from` to reflect the
post-mutation PackageIds. Add a regression test that runs
`find_by_id` on every dependency after a rewrite.

Also update the release-notes install snippets to use `mktemp -d` +
`trap` cleanup so the downloaded tarball and extracted binaries are
removed after installation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: skip remote-index URL check in satisfies() for URL preservation
b74f184 
URL preservation keeps the previous registry URL in the lockfile, which
won't match the current UV_DEFAULT_INDEX when mirrors differ. The
satisfies() check was returning MissingRemoteIndex on every run,
forcing a full re-resolve even when nothing changed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: normalize newly added packages to canonical registry URL
cd50045 
When `uv add` introduces a new package, it gets the current
UV_DEFAULT_INDEX mirror URL as its registry source. Previously,
rewrite_urls_from only preserved URLs for packages matching by
(name, version) — new packages kept the mirror URL.

Now infer the canonical registry URL from the previous lockfile and
apply it to all registry packages, including newly added ones, so
the lockfile stays consistent regardless of which mirror resolved
the package.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: replace previous-lock URL inference with UV_PYPI_PROXIES env var
6304dc2 
Replace the complex "infer canonical URL from previous lockfile" logic
with a simple env var mapping. UV_PYPI_PROXIES maps canonical URLs to
proxy URLs (format: canonical:proxy,canonical2:proxy2).

After resolution, rewrite_proxy_urls() replaces every matching proxy
registry URL with its canonical counterpart. This handles both existing
and newly added packages uniformly — no need to track the previous
lockfile at all.

Example:
  UV_PYPI_PROXIES=https://pypi.org/simple:https://pypi-proxy.dev.databricks.com/simple

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: simplify mod.rs — restore upstream code, inject canonical URLs
e601821 
Instead of commenting out the entire remotes collection and check in
satisfies(), restore the original upstream code and just add canonical
URLs from UV_PYPI_PROXIES to the remotes set. This reduces the mod.rs
diff from ~60 lines of comments to 8 added lines.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: replace internal URL with example.com in docs
0b3e70b 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: fix UrlString construction — use SmallString::from for &str
0755589 
UrlString doesn't implement From<&str>; use UrlString::new(SmallString::from(...))
to construct from string slices.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork-release: add UV_PYPI_PROXIES setup instructions to release notes
16af18d 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: rename UV_PYPI_PROXIES to UV_INDEX_PROXIES
c24b038 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: also rewrite proxy URLs on Unchanged lock results
895fc45 
When satisfies() returns Satisfied (lockfile already up-to-date), the
result is Unchanged and the rewrite_proxy_urls code path was skipped.
Now handle both Changed and Unchanged cases so proxy URLs get rewritten
to canonical even when no re-resolution was needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: resolve canonical URLs back to proxy for uv sync
b062568 
The lockfile stores canonical URLs (e.g. pypi.org/simple), but uv sync
needs to fetch from the proxy that is actually reachable. Add proxy_url()
reverse mapping and apply it at the 3 points where RegistrySource::Url
is converted to IndexUrl for HTTP fetching.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: add debug logging for proxy URL resolution
49be384 
Log when rewriting proxy→canonical (lock) and canonical→proxy (sync)
so the mapping is visible in verbose mode (`uv -v`).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: apply UV_INDEX_PROXIES to default index for build resolution
26ce1c5 
When no explicit default index is configured, uv falls back to
pypi.org/simple. If UV_INDEX_PROXIES maps pypi.org to a proxy, the
default index now resolves through the proxy so that build-system.requires
(e.g. setuptools) can be fetched without direct pypi.org access.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: reduce proxy URL log noise — use trace for per-package, debug f…
b9eb485 
…or summary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy @claude
fork: apply UV_INDEX_PROXIES to IndexUrls::default_index() too
475d6e5 
The previous fix only patched IndexLocations::default_index(), but the
resolver uses IndexUrls::default_index() which had a separate fallback
to the hardcoded PyPI URL. Apply the same proxy mapping here so that
uv lock also fetches through the proxy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harupy harupy closed this Apr 28, 2026
@harupy harupy mentioned this pull request May 18, 2026
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@harupy