Skip to content

Refactor DAG internals and lazy-load networkx#830

Merged
tobiasraabe merged 13 commits intomainfrom
refactor/internal-dag-networkx-optional
Mar 27, 2026
Merged

Refactor DAG internals and lazy-load networkx#830
tobiasraabe merged 13 commits intomainfrom
refactor/internal-dag-networkx-optional

Conversation

@tobiasraabe
Copy link
Copy Markdown
Member

@tobiasraabe tobiasraabe commented Mar 26, 2026

Summary

  • replace the internal networkx DAG usage with a pytask-owned DAG implementation
  • keep build_dag() and the visualization path returning a networkx.DiGraph via on-demand conversion
  • make networkx optional and only import it on the DAG export boundary

Benchmarks

Compared this branch against origin/main in an isolated worktree after plain uv sync in both trees.

  • pytask --help
    • current: 282.3 ms ± 43.0 ms
    • origin/main: 339.8 ms ± 27.3 ms
    • about 1.20x faster
  • python -c "import pytask"
    • current: 284.5 ms ± 40.5 ms
    • origin/main: 320.1 ms ± 26.8 ms
    • about 1.13x faster

The current branch runtime environment had no installed networkx after uv sync, while origin/main still did.

Testing

  • just typing
  • uvx ruff check src/_pytask/dag.py src/_pytask/dag_command.py src/_pytask/dag_graph.py src/_pytask/dag_utils.py src/_pytask/execute.py src/_pytask/session.py src/_pytask/shared.py src/_pytask/mark/__init__.py tests/test_dag_command.py tests/test_dag_utils.py pyproject.toml docs/source/tutorials/visualizing_the_dag.md
  • uvx ruff format --check src/_pytask/dag.py src/_pytask/dag_command.py src/_pytask/dag_graph.py src/_pytask/dag_utils.py src/_pytask/execute.py src/_pytask/session.py src/_pytask/shared.py src/_pytask/mark/__init__.py tests/test_dag_command.py tests/test_dag_utils.py
  • uv run --group test pytest tests/test_dag_command.py tests/test_dag_utils.py tests/test_build.py tests/test_execute.py tests/test_mark.py tests/test_dag.py -q
  • uv run --group test pytest tests/test_collect_command.py tests/test_execute.py tests/test_lockfile.py tests/test_profile.py -q

@tobiasraabe tobiasraabe enabled auto-merge March 27, 2026 22:46
@tobiasraabe tobiasraabe disabled auto-merge March 27, 2026 22:48
@tobiasraabe tobiasraabe added this pull request to the merge queue Mar 27, 2026
Merged via the queue into main with commit 8a86c8a Mar 27, 2026
21 checks passed
@tobiasraabe tobiasraabe deleted the refactor/internal-dag-networkx-optional branch March 27, 2026 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant