From 0fd24f3ed7663443488e87ca1f0ad77e4508269f Mon Sep 17 00:00:00 2001 From: Andy Grove Date: Sat, 23 May 2026 19:12:13 -0600 Subject: [PATCH 1/2] docs(release): cover TestPyPI verification and fix stale wheel-download notes - Add a 'Verify the Python wheels from TestPyPI' subsection under Verifying Release Candidates so RC verifiers can install from test.pypi.org if the release manager has done the pre-vote dry-run. - Document that download-python-wheels.py needs an interactive terminal for gpg signing; the wheels download fine before the signing step, so a TestPyPI-only dry-run can ignore the traceback. - Replace the manylinux_2_17 wheel filenames in the expected-files list with glibc-version-agnostic globs; the runner images advance their glibc tag. - Drop the 'build-sdist does not upload its output' warning (made stale by #1753) and fix the local-rebuild fallback to use 'maturin sdist' instead of the broken 'maturin build --release --sdist' (fixed in #1759, which produced both a wheel and an sdist). - Refresh the per-platform fallback artifact names: dist-windows-2022 (#1751) and the new dist-sdist artifact. - Note that the smoke-test venv needs Python >= 3.10; macOS's stock python3 (3.9) yields a misleading 'No matching distribution' error against the cp310-abi3 wheels. --- dev/release/README.md | 65 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 10 deletions(-) diff --git a/dev/release/README.md b/dev/release/README.md index b19e3b0e8..021f53df2 100644 --- a/dev/release/README.md +++ b/dev/release/README.md @@ -203,12 +203,42 @@ For the release to become "official" it needs at least three PMC members to vote ### Verifying Release Candidates -The `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like: +The `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. It downloads the source tarball from the ASF dev SVN, verifies the GPG signature and checksums, and builds the Rust workspace. Run it like: ``` ./dev/release/verify-release-candidate.sh 0.11.0 0 ``` +#### (Optional) Verify the Python wheels from TestPyPI + +If the release manager has uploaded the RC's Python wheels to +[test.pypi.org](https://test.pypi.org/project/ballista/) as part of their +pre-vote dry-run (see [Publish Python Wheels to PyPI](#publish-python-wheels-to-pypi) +below), verifiers can install them in a throwaway virtualenv to sanity-check +the artifacts that will ship to real PyPI. The wheels there are byte-identical +to what would be uploaded to pypi.org if the vote passes. + +The wheels are built as `cp310-abi3`, so the venv needs Python ≥ 3.10: + +```bash +export BALLISTA_VERSION=0.11.0 # version under vote + +python3.10 -m venv /tmp/ballista-rc-verify +source /tmp/ballista-rc-verify/bin/activate +pip install -i https://test.pypi.org/simple/ \ + --extra-index-url https://pypi.org/simple/ \ + ballista==${BALLISTA_VERSION} +python -c "from ballista import BallistaSessionContext; print('ok')" +deactivate +``` + +`--extra-index-url` is required because TestPyPI does not mirror dependencies +like `pyarrow` and `datafusion`. + +If no version of `ballista==${BALLISTA_VERSION}` is yet on TestPyPI, the +release manager has not (yet) uploaded the dry-run — that step is +optional and not a blocker for voting. + #### If the release is not approved If the release is not approved, fix whatever the problem is, merge changelog @@ -363,11 +393,22 @@ python ../dev/release/download-python-wheels.py ${BALLISTA_VERSION}-rc${BALLISTA ls *.whl *.tar.gz # confirm filenames carry the right version ``` +> **GPG signing needs an interactive terminal.** The script signs each +> artifact with `gpg --detach-sig`, which prompts for the key passphrase. From +> a non-interactive shell the prompt fails with `gpg: signing failed: +> Inappropriate ioctl for device` and the script aborts after the first +> artifact. Either run from an interactive shell, or configure `gpg-agent` +> with `pinentry-mode loopback` and a cached passphrase. The wheels and sdist +> are downloaded before the signing step, so for a TestPyPI-only dry-run the +> traceback is harmless (PyPI does not accept `.asc` files anyway). + The merged artifact should contain one of each of the following platform wheels -(file naming uses [PEP 425](https://peps.python.org/pep-0425/) tags): +(file naming uses [PEP 425](https://peps.python.org/pep-0425/) tags; the +`manylinux_X_Y` glibc tag depends on the Linux runner image and changes over +time, so glob it rather than pinning a specific value): -- `ballista-${BALLISTA_VERSION}-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl` -- `ballista-${BALLISTA_VERSION}-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl` +- `ballista-${BALLISTA_VERSION}-cp310-abi3-manylinux_*_x86_64.whl` +- `ballista-${BALLISTA_VERSION}-cp310-abi3-manylinux_*_aarch64.whl` - `ballista-${BALLISTA_VERSION}-cp310-abi3-macosx_*_arm64.whl` - `ballista-${BALLISTA_VERSION}-cp310-abi3-win_amd64.whl` - `ballista-${BALLISTA_VERSION}.tar.gz` (sdist) @@ -383,21 +424,22 @@ The merged artifact should contain one of each of the following platform wheels > --name dist-manylinux-aarch64 \ > --name dist-manylinux-x86_64 \ > --name dist-macos-latest \ -> --name dist-windows-latest +> --name dist-windows-2022 \ +> --name dist-sdist > ``` > > Then re-sign each downloaded file with `gpg --detach-sig` and regenerate the > `.sha256` / `.sha512` checksums the same way `download-python-wheels.py` > does. Do **not** proceed to upload an incomplete platform set. > -> The `build-sdist` job currently does not upload its output as a workflow -> artifact, so the sdist may be absent from the merged `dist` even when all -> wheels are present. If you need the sdist, build it locally from the RC tag: +> If only the sdist is missing, it can also be rebuilt locally from the RC +> tag (the `build-sdist` job uploads it as `dist-sdist` so this should not +> normally be needed): > > ```bash > git checkout ${BALLISTA_VERSION}-rc${BALLISTA_RC_NUM} > cd python -> uv run --no-project maturin build --release --sdist --out dist +> uv run --no-project maturin sdist --out dist > ``` #### Validate the Artifacts @@ -420,7 +462,10 @@ the common ways a release goes wrong. ```bash twine upload --repository testpypi *.whl *.tar.gz -python -m venv /tmp/ballista-pypi-smoke +# Wheels are cp310-abi3 so the venv needs Python >= 3.10. Using `python -m venv` +# with macOS's stock /usr/bin/python3 (3.9) silently picks no wheel and pip +# reports a misleading "No matching distribution found". +python3.10 -m venv /tmp/ballista-pypi-smoke source /tmp/ballista-pypi-smoke/bin/activate pip install -i https://test.pypi.org/simple/ \ --extra-index-url https://pypi.org/simple/ \ From 31e30e5a2126a7fb9e03d11d98c2aac533854719 Mon Sep 17 00:00:00 2001 From: Andy Grove Date: Sat, 23 May 2026 19:19:46 -0600 Subject: [PATCH 2/2] docs(release): use 53.0.0 in the TestPyPI verify example --- dev/release/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dev/release/README.md b/dev/release/README.md index 021f53df2..2b43e77b5 100644 --- a/dev/release/README.md +++ b/dev/release/README.md @@ -221,7 +221,7 @@ to what would be uploaded to pypi.org if the vote passes. The wheels are built as `cp310-abi3`, so the venv needs Python ≥ 3.10: ```bash -export BALLISTA_VERSION=0.11.0 # version under vote +export BALLISTA_VERSION=53.0.0 # version under vote python3.10 -m venv /tmp/ballista-rc-verify source /tmp/ballista-rc-verify/bin/activate