Skip to content

Use nextest for CI test jobs #22544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Dandandan wants to merge 1 commit into
base: main
Choose a base branch
from
+51 −17
Draft

Use nextest for CI test jobs #22544

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions .github/workflows/extended.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ permissions:
jobs:
# Run extended tests (with feature 'extended_tests')
linux-test-extended:
name: cargo test 'extended_tests' (amd64)
name: cargo nextest 'extended_tests' (amd64)
runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=32,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }}
# note: do not use amd/rust container to preserve disk space
steps:
Expand All @@ -80,16 +80,20 @@ jobs:
run: |
sudo apt-get update
sudo apt-get install -y protobuf-compiler
- name: Install nextest
uses: taiki-e/install-action@213ccc1a076163c093f914550b94feb90fab916d # v2.79.2
with:
tool: nextest
Copy link
Copy Markdown
Contributor

@alamb alamb May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a nice way to avoid having to rebuild nextest each run

# For debugging, test binaries can be large.
- name: Show available disk space
run: |
df -h
- name: Run tests (excluding doctests)
- name: Run tests with nextest (excluding doctests)
env:
RUST_BACKTRACE: 1
run: |
cargo test \
--profile ci \
cargo nextest run \
--cargo-profile ci \
--exclude datafusion-examples \
--exclude datafusion-benchmarks \
--exclude datafusion-cli \
Expand All @@ -105,7 +109,7 @@ jobs:

# Check answers are correct when hash values collide
hash-collisions:
name: cargo test hash collisions (amd64)
name: cargo nextest hash collisions (amd64)
runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=16,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }}
container:
image: amd64/rust
Expand All @@ -120,10 +124,14 @@ jobs:
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- name: Install nextest
uses: taiki-e/install-action@213ccc1a076163c093f914550b94feb90fab916d # v2.79.2
with:
tool: nextest
- name: Run tests
run: |
cd datafusion
cargo test --profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --exclude datafusion-sqllogictest --exclude datafusion-cli --workspace --lib --tests --features=force_hash_collisions,avro
cargo nextest run --cargo-profile ci --exclude datafusion-examples --exclude datafusion-benchmarks --exclude datafusion-sqllogictest --exclude datafusion-cli --workspace --lib --tests --features=force_hash_collisions,avro
cargo clean

sqllogictest-sqlite:
Expand All @@ -144,4 +152,4 @@ jobs:
apt-get update && apt-get install -y protobuf-compiler
- name: Run sqllogictest
run: |
cargo test --features backtrace,parquet_encryption --profile ci-optimized --test sqllogictests -- --include-sqlite
cargo test --features backtrace,parquet_encryption --profile ci-optimized --test sqllogictests -- --include-sqlite
46 changes: 36 additions & 10 deletions .github/workflows/rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -268,7 +268,7 @@ jobs:

# Library and integration tests
linux-test:
name: cargo test (amd64)
name: cargo nextest (amd64)
needs: linux-build-lib
runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=16,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }}
container:
Expand All @@ -290,12 +290,16 @@ jobs:
with:
save-if: ${{ github.ref_name == 'main' }}
shared-key: "amd-ci"
- name: Run tests (excluding doctests and datafusion-cli)
- name: Install nextest
uses: taiki-e/install-action@213ccc1a076163c093f914550b94feb90fab916d # v2.79.2
with:
tool: nextest
- name: Run tests with nextest (excluding doctests and datafusion-cli)
env:
RUST_BACKTRACE: 1
run: |
cargo test \
--profile ci \
cargo nextest run \
--cargo-profile ci \
--exclude datafusion-examples \
--exclude ffi_example_table_provider \
--exclude datafusion-cli \
Expand All @@ -304,6 +308,16 @@ jobs:
--tests \
--bins \
--features serde,avro,json,backtrace,integration-tests,parquet_encryption,substrait
- name: Run sqllogictests
env:
RUST_BACKTRACE: 1
run: |
# cargo-nextest lists this custom harness but does not execute it.
cargo test \
--profile ci \
--package datafusion-sqllogictest \
--test sqllogictests \
--features backtrace,parquet_encryption,substrait
- name: Verify Working Directory Clean
run: git diff --exit-code
# Check no temporary directories created during test.
Expand All @@ -318,7 +332,7 @@ jobs:

# datafusion-cli tests
linux-test-datafusion-cli:
name: cargo test datafusion-cli (amd64)
name: cargo nextest datafusion-cli (amd64)
needs: linux-build-lib
runs-on: ${{ vars.USE_RUNS_ON == 'true' && format('runs-on={0},family=m8a+m7a+c8a,cpu=16,image=ubuntu24-full-x64,extras=s3-cache,disk=large,tag=datafusion', github.run_id) || 'ubuntu-latest' }}
steps:
Expand All @@ -334,15 +348,19 @@ jobs:
with:
save-if: false # set in linux-test
shared-key: "amd-ci"
- name: Run tests (excluding doctests)
- name: Install nextest
uses: taiki-e/install-action@213ccc1a076163c093f914550b94feb90fab916d # v2.79.2
with:
tool: nextest
- name: Run tests with nextest (excluding doctests)
env:
RUST_BACKTRACE: 1
AWS_ENDPOINT: http://127.0.0.1:9000
AWS_ACCESS_KEY_ID: TEST-DataFusionLogin
AWS_SECRET_ACCESS_KEY: TEST-DataFusionPassword
TEST_STORAGE_INTEGRATION: 1
AWS_ALLOW_HTTP: true
run: cargo test --features backtrace --profile ci -p datafusion-cli --lib --tests --bins
run: cargo nextest run --features backtrace --cargo-profile ci -p datafusion-cli --lib --tests --bins
- name: Verify Working Directory Clean
run: git diff --exit-code

Expand Down Expand Up @@ -457,6 +475,10 @@ jobs:
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- name: Install nextest
uses: taiki-e/install-action@213ccc1a076163c093f914550b94feb90fab916d # v2.79.2
with:
tool: nextest
- name: Generate benchmark data and expected query results
run: |
mkdir -p datafusion/sqllogictest/test_files/tpch/data
Expand All @@ -470,7 +492,7 @@ jobs:
# increase stack size to fix stack overflow
export RUST_MIN_STACK=20971520
export TPCH_DATA=`realpath datafusion/sqllogictest/test_files/tpch/data`
cargo test plan_q --package datafusion-benchmarks --profile ci --features=ci -- --test-threads=1
cargo nextest run --package datafusion-benchmarks --cargo-profile ci --features=ci --test-threads=1 plan_q
INCLUDE_TPCH=true cargo test --features backtrace,parquet_encryption,substrait --profile ci --package datafusion-sqllogictest --test sqllogictests
- name: Verify Working Directory Clean
run: git diff --exit-code
Expand Down Expand Up @@ -562,7 +584,7 @@ jobs:
# historically came from datafusion-benchmarks (now covered on amd64) or
# flaky sqllogictest metrics.
macos-aarch64:
name: cargo test (macos-aarch64)
name: cargo nextest (macos-aarch64)
runs-on: macos-15
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
Expand All @@ -571,9 +593,13 @@ jobs:
fetch-depth: 1
- name: Setup Rust toolchain
uses: ./.github/actions/setup-macos-aarch64-builder
- name: Install nextest
uses: taiki-e/install-action@213ccc1a076163c093f914550b94feb90fab916d # v2.79.2
with:
tool: nextest
- name: Run datafusion-ffi tests
shell: bash
run: cargo test --profile ci -p datafusion-ffi --lib --tests --features integration-tests
run: cargo nextest run --cargo-profile ci -p datafusion-ffi --lib --tests --features integration-tests

vendor:
name: Verify Vendored Code
Expand Down
Loading