feat: auto-derive idempotency keys on @queue.task#143
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Turns the existing
unique_keydeduplication infrastructure (already presentend-to-end in SQLite, Postgres, and Redis with partial unique indexes / Lua
scripts) into a first-class, opt-in task feature. No Rust changes, no schema
migration, no new Storage methods — pure-Python wiring.
auto:{sha256(task_name|serialized_payload)[:32]}. Hashedafter the task's serializer runs, so the wire-payload contract is what
decides equivalence.
idempotency_key="…"overrides the auto-derived key.idempotent=Falsedisables auto-derivation for one submission evenon a task registered with
idempotent=True.unique_keyparameter kept as a documented alias for back-compat.enqueue_many()viaidempotency_keys: list[str | None]and auniform
idempotentflag.on_enqueuemiddleware sees and can mutate the new keys.Slot release semantics are unchanged — the dedup window is held only while the
job is
pendingorrunning; once it terminates, a fresh call with the samearguments creates a new job. This is enforced by the existing partial unique
index, not new code.
Files
py_src/taskito/app.py—_task_idempotentmap,_resolve_unique_key()helper, new kwargs on
enqueue()/enqueue_many().py_src/taskito/mixins/decorators.py—idempotent: bool = Falseon@queue.task().py_src/taskito/task.py—TaskWrapper.apply_async()plumbs the new kwargs.tests/python/test_idempotent.py— 8 new tests.docs/content/docs/guides/reliability/idempotency.mdx+ meta.json — guide.Test plan
uv run python -m pytest tests/python/test_idempotent.py tests/python/test_unique.py -v(12 passed)uv run python -m pytest tests/python/ -x(506 passed, 9 skipped)uv run ruff check py_src/ tests/python/test_idempotent.pyuv run mypy py_src/taskito/ --no-incremental(clean across 121 files)cargo check --workspace(clean)