v0.4.0rc1

This is the first release candidate for the v0.4.* release series.

The final release still requires some changes. For example, the documentation needs to be extended. But, the essential parts are already there, and it is time to collect some final feedback! Let me know what you think and what needs to be improved. You can comment in the discussion for this release #422.

To install the pre-release, use

$ pip install pytask --pre
$ conda install -c "conda-forge/label/pytask_rc" pytask

Now, let's take a look at the changes.

What's Changed

New

• Dependencies and products of tasks have new interfaces that are explained in this tutorial.
• You can also now declare products by allowing task functions to return. Follow this guide.
• If you have inputs to task functions that should be hashed to detect any changes, follow this guide.
• Before, only pathlib.Paths received special treatment as dependencies or products to task functions. Now, it is possible to define your own nodes that simplify, for example, loading pickle files as this guide explains. But many more extensions are possible, like defining data in an S3 bucket as a dependency or product.
• The functional interface has been reworked and now accepts tasks directly, allowing you to execute pytask on the command line or in Jupyter notebooks. The documentation must still be written, but here is your starting point.

Removals

• Python 3.7 is no longer supported.
• @pytask.mark.parametrize is removed. Follow this tutorial instead.

Deprecations

• @pytask.mark.depends_on, @pytask.mark.produces are deprecated and will be removed in v0.5.0.
• @pytask.mark.task is deprecated. Use @pytask.task instead.
• Paths defined as strings are deprecated and should be replaced with proper pathlib.Path objects.

Full list of changes

• Remove Python 3.7 support and add a new action for mamba. by @tobiasraabe in #323
• Replace pony with sqlalchemy>=1.4.36. by @tobiasraabe in #387
• Remove @pytask.mark.parametrize. by @tobiasraabe in #391
• Parse dependencies from all args if depends_on is not used. by @tobiasraabe in #384
• Add products with typing.Annotation. by @tobiasraabe in #394
• Refactor pybaum to _pytask.tree_util. by @tobiasraabe in #395
• Replace pybaum with optree and add paths to PythonNode names. by @tobiasraabe in #396
• Add support for NamedTuple and attrs classes in @pytask.mark.task(kwargs=...). by @tobiasraabe in #397
• Deprecate decorators for depends_on and produces. by @tobiasraabe in #398
• Use protocols instead of ABCs. by @tobiasraabe in #402
• Allow tasks to return products. by @tobiasraabe in #404
• Tracking changes in v0.4.0. by @tobiasraabe in #400
• Bump peter-evans/create-pull-request from 5.0.1 to 5.0.2 by @dependabot in #390
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #388
• Allow to use prefix trees as nodes to parse function returns. by @tobiasraabe in #406
• Remove .value from Node protocol. by @tobiasraabe in #408
• Make .from_annot an optional feature of nodes. by @tobiasraabe in #409
• Allow to pass functions to PythonNode(hash=...). by @tobiasraabe in #410
• Add protocols for tasks. by @tobiasraabe in #412
• Remove scripts to generate .svgs. by @tobiasraabe in #413
• Allow more ruff rules. by @tobiasraabe in #414
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #407
• A new functional interface. by @tobiasraabe in #411
• Deprecate @pytask.mark.task in favor of @pytask.task. by @tobiasraabe in #417
• Simplify and fix code in dag.py. by @tobiasraabe in #418
• Convert DeprecationWarning to FutureWarning for deprecated decorators. by @tobiasraabe in #420
• Remove deprecation warning for produces. by @tobiasraabe in #421
• Document new interface. by @tobiasraabe in #392
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #415

Full Changelog: v0.3.2...v0.4.0rc1

v0.3.2

Highlights

This release contains the following highlights:

• Previously, if you accidentally hit the save button on an unchanged task file, the task would be rerun by pytask, although nothing had changed. Now, pytask wouldn't rerun the task because it also compares the hashes of task files, not only the modification timestamp.
• If you want to enforce rerunning tasks, there is now a --force flag. Take the function name/id of the task and run pytask -k <task id> --force, and the task + its necessary tasks will be executed. Or delete a product from the task you want to rerun.
• The import mechanism for task modules has been reworked, and errors resolved. Thanks to @NickCrews!

Additionally, the @pytask.mark.parametrize decorator is deprecated and will be removed in pytask v0.4. If you use the decorator, you will have two options:

1. (Recommended) Upgrade your code to the new approach for repeating tasks described in this tutorial.
2. Or, pin pytask to pytask<0.4 and silence the deprecation warning by setting silence_parametrize_deprecation = true in your pyproject.toml under [tool.pytask.ini_options].

What's Changed

• Update version numbers in animations. by @tobiasraabe in #345
• Add dependabot for GitHub actions. by @tobiasraabe in #348
• Publish db. by @tobiasraabe in #352
• Refactor nodes. by @tobiasraabe in #355
• Add -f/--force to force executing tasks. by @tobiasraabe in #354
• Add hashing to avoid re-executing tasks when modification times changed. by @tobiasraabe in #357
• Update update_plugin_list.py. by @tobiasraabe in #364
• Rework panel with sphinx-design. by @tobiasraabe in #365
• Add light and dark logos for the documentation. by @tobiasraabe in #366
• Fix the panel on the index page of the documentation. by @tobiasraabe in #367
• Fix error introduced in #364. by @tobiasraabe in #369
• Revert change turning Node.state() into a hook. by @tobiasraabe in #370
• Rename Node back to MetaNode. by @tobiasraabe in #371
• Clearer documentation for pytask dag -o. by @tobiasraabe in #376
• Conditionally skip tests on MacOS. by @tobiasraabe in #378
• Deprecate @pytask.mark.parametrize. by @tobiasraabe in #381
• Fix the import mechanism for task modules. by @NickCrews in #373
• Update changes. by @tobiasraabe in #383

New Contributors

• @dependabot made their first contribution in #349
• @NickCrews made their first contribution in #373

Full Changelog: v0.3.1...v0.3.2

v0.3.1

What's Changed

• Fix bug when passing no path on the command line. by @tobiasraabe in #337
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #341
• [automated] Update plugin list by @github-actions in #340

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Highlights

This release includes a breaking change due to internal refactorings. The change affects how command line options and the configuration file are loaded and validated. For users, the changes are subtle; the help pages of the commands have prettier options and default values.

Make sure to upgrade pytask and the plugins to v0.3 or pin the packages to <0.3.

There is some delay until the updates for pytask and its plugins are available. Be aware of errors when using mixed v0.2 and v0.3 installations.

The most significant benefit is for developers who want to add command line options and configuration values. The parsing can now be handled with proper click types, for example, EnumChoice to implement choice options. Defaults are attached to command line options and are automatically displayed in the help pages.

What's Changed

• Update workflow status badge. by @tobiasraabe in #326
• Deprecate INI configurations. by @tobiasraabe in #313
• Add ruff to linters. by @tobiasraabe in #329
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #328
• Refactor database.py. by @tobiasraabe in #332
• Add a guide to migrate from scripts to pytask. by @tobiasraabe in #330
• Upgrade attrs and related code. by @tobiasraabe in #333
• Use ruff with all rules selected by default. by @pre-commit-ci in #331
• Set target version for ruff. by @tobiasraabe in #334
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #335
• Extend the API documentation. by @tobiasraabe in #336

Full Changelog: v0.2.7...v0.3.0

v0.2.7

News

This is probably the last patch release before the v0.3.0 release. There are some highlights like the new terminal animations in the documentation. I hope you enjoy them.

What's Changed

• Replace pydot with pygraphviz. by @tobiasraabe in #308
• Add refurb to pre-commit hooks. by @tobiasraabe in #311
• Fix links in docs. by @tobiasraabe in #310
• Add Python 3.11 to CI. by @tobiasraabe in #307
• Clarify example in docs on nested dependencies and products. by @tobiasraabe in #318
• Formalize more choice options to enums. by @tobiasraabe in #321
• Add terminal animations. by @tobiasraabe in #322
• Collect automatically created tasks. by @tobiasraabe in #325

Full Changelog: v0.2.6...v0.2.7

v0.2.6

What's Changed

• Add docformatter. by @tobiasraabe in #297
• Move and publish some functions related to warnings. by @tobiasraabe in #298
• [automated] Update plugin list by @github-actions in #300
• fix[docs]: Fix mismatched backticks by @augustebaum in #303
• Remove colorama. by @tobiasraabe in #305
• Fix typo in link and adjust documentation for warnings. by @hmgaudecker in #306

New Contributors

Many thanks to the new contributors 🙇!

• @augustebaum made their first contribution in #303
• @hmgaudecker made their first contribution in #306

Full Changelog: v0.2.5...v0.2.6

v0.2.5

Highlight

The highlight of this release is a dry-run mode that shows which tasks will be run during the next execution. Just type pytask --dry-run. Here is the documentation.

What's Changed

• Pin pybaum to >=0.1.1. by @tobiasraabe in #288
• pytask collect displays shortened task names. by @tobiasraabe in #289
• Implement a dry-run mode. by @tobiasraabe in #290
• Fix bug with --pdb and pytask-latex. by @tobiasraabe in #296

Full Changelog: v0.2.4...v0.2.5

v0.2.4

What's Changed

• Some pages of the documentation have been improved. by @tobiasraabe in #279 and #281
• Don't display traceback with 'previous task failed' with verbose <= 1. by @tobiasraabe in #285
• Show all files that are not produced by a task. by @tobiasraabe in #287

Full Changelog: v0.2.3...v0.2.4

v0.2.3

Highlights

This release includes some minor fixes. For example, deprecation warnings are not activated by default anymore, and you can remove the filters for these warnings again.

What's Changed

• Fix pytask clean when git is not installed. by @tobiasraabe in #276
• Do not capture DeprecationWarning and PendingDeprecationWarning by default. by @tobiasraabe in #277
• Reduce the number of locations where warnings are raised. by @tobiasraabe in #278

Full Changelog: v0.2.2...v0.2.3

v0.2.2

Highlight

The command pytask clean has been reworked! 🎉

pytask clean allows to clean your project from obsolete files. For example, when you renamed a product of a task and the old product still exists in the build folder.

All files which are not known by pytask or git can be cleaned.

You find more information in the tutorial.

What's Changed

• Fix info under live execution table. by @tobiasraabe in #267
• Rework the clean command. by @tobiasraabe in #273

Full Changelog: v0.2.1...v0.2.2