[ENG-2386] CI tweaks to help speed up tests

[johnewart]

Ticket ENG-2386

Description Of Changes

Disable JIT in CI to avoid the overhead; which can be problematic on resource-constrained systems. Also changed xdist to use loadfile to see if that helps reduce contention and cut down the number of xdist workers to 2 to avoid resource contention

Code Changes

See above

Steps to Confirm

None, test pipeline changes

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Updates unreleased work already in Changelog, no new entry necessary
• UX feedback:
  • All UX related changes have been reviewed by a designer
  • No UX review needed
• Followup issues:
  • Followup issues created
  • No followup issues
• Database migrations:
  • Ensure that your downrev is up to date with the latest revision on main
  • Ensure that your downgrade() migration is correct and works
    • If a downgrade migration is not possible for this change, please call this out in the PR description!
  • No migrations
• Documentation:
  • Documentation complete, PR opened in fidesdocs
  • Documentation issue created in fidesdocs
  • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
  • No documentation updates required

Summary by CodeRabbit

• Chores
  • Improved CI/CD infrastructure with dynamic parallel test execution controlled via environment variables, enabling optimized resource utilization and faster feedback.
  • Enhanced test distribution strategies with multiple execution modes and performance-focused CI environment settings.
  • Implemented targeted optimizations to reduce build time and accelerate feedback cycles throughout the development workflow.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored		Mar 5, 2026 5:38am
fides-privacy-center	Ignored		Mar 5, 2026 5:38am

[johnewart]

Testing this in CI so opening a PR, may take a few iterations - I will ping you when it's ready for review 😄

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b4b058cc-6e85-4755-a15c-cd54e2818731

📥 Commits

Reviewing files that changed from the base of the PR and between f7b17b5 and b36c03e.

📒 Files selected for processing (4)

• .github/workflows/backend_checks.yml
• noxfiles/ci_nox.py
• noxfiles/constants_nox.py
• noxfiles/setup_tests_nox.py

---

📝 Walkthrough

Walkthrough

These changes enable environment-variable-driven pytest parallelism configuration and Python JIT control. Modifications add PYTEST_XDIST_WORKERS environment variable support in CI workflows, update nox configuration to source this variable, introduce conditional Python JIT disabling in CI environments, and extend XdistConfig with a distribution mode attribute for pytest-xdist.

Changes

Cohort / File(s)	Summary
Pytest Parallelism Configuration .github/workflows/backend_checks.yml, noxfiles/ci_nox.py, noxfiles/setup_tests_nox.py	Added environment-variable-driven parallelism support: PYTEST_XDIST_WORKERS env var in workflow (defaulting to "auto"), dynamic sourcing in ci_nox.py, and new optional dist_mode attribute in XdistConfig for specifying pytest-xdist distribution strategy (e.g., "loadfile").
CI Environment Configuration noxfiles/constants_nox.py	Introduced PYTHON_JIT_OFF constant that conditionally sets PYTHON_JIT=0 when running in CI, integrated into EXEC_UV command tuple.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Parallel workers hop with glee,
Configuration flows so free,
JIT turned off in CI's domain,
Tests run swift through the code's terrain!
thump thump ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: CI optimizations to improve test speed by reducing overhead and resource contention.
Description check	✅ Passed	The PR description covers the key sections including Description of Changes with specific technical details, Code Changes reference, Steps to Confirm, and includes the Pre-Merge Checklist template with appropriate items selected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

• 📝 Generate docstrings (stacked PR)
• 📝 Generate docstrings (commit on current branch)

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch johnewart/ENG-2386

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[greptile-apps]

Greptile Summary

This PR makes three targeted CI performance improvements: it caps xdist parallel workers at 2 for the Safe-Tests job, switches to loadfile distribution mode to keep tests in the same file on the same worker (reducing fixture overhead and resource contention), and disables the Python 3.13 JIT in CI via PYTHON_JIT=0 to avoid cold-cache startup overhead on resource-constrained runners.

• PYTEST_XDIST_WORKERS: "2" is set only on the Safe-Tests workflow job; other jobs (external datastores, SaaS connectors) already explicitly disable xdist or run single-threaded, so they are unaffected.
• PYTHON_JIT_OFF is correctly scoped to the EXEC_UV tuple (which drives all uv run pytest invocations) and is a no-op on Python versions prior to 3.13.
• The --dist guard in XdistConfig.args (parallel_runners != "0") correctly prevents the --dist loadfile flag from being emitted when xdist is disabled, avoiding an invalid CLI combination.

Confidence Score: 5/5

• Safe to merge — changes are limited to CI configuration and nox helper files with no impact on production code.
• All changes are purely in CI tooling (GitHub Actions workflow + noxfiles). The logic is straightforward, edge cases (xdist disabled, non-3.13 Python) are handled correctly, and the changes are backward-compatible (defaults preserve original behavior when env vars are absent).
• No files require special attention.

Important Files Changed

Filename	Overview
.github/workflows/backend_checks.yml	Adds PYTEST_XDIST_WORKERS: "2" env var to the Safe-Tests job to cap parallel test workers; scoped only to this job by design.
noxfiles/ci_nox.py	Reads PYTEST_XDIST_WORKERS from environment (defaulting to "auto") and adds dist_mode="loadfile" to reduce fixture/resource contention across workers.
noxfiles/constants_nox.py	Adds PYTHON_JIT_OFF constant that sets PYTHON_JIT=0 when running in CI, then splices it into the EXEC_UV docker exec command tuple.
noxfiles/setup_tests_nox.py	Adds optional dist_mode field to XdistConfig and guards --dist flag insertion so it is only emitted when parallel_runners != "0".

_{Last reviewed commit: b36c03e}