Skip to content

fix(app): correct enqueue_many middleware args and option propagation#106

Merged
pratyush618 merged 1 commit into
masterfrom
fix/enqueue-many-middleware
May 2, 2026
Merged

fix(app): correct enqueue_many middleware args and option propagation#106
pratyush618 merged 1 commit into
masterfrom
fix/enqueue-many-middleware

Conversation

@pratyush618
Copy link
Copy Markdown
Collaborator

Summary

P0-3 from the pre-release audit. enqueue_many had three bugs in the on_enqueue middleware dispatch that the test suite was missing entirely:

  1. Always passed args_list[0] and kw_list[0] to every middleware call, so middleware could not distinguish jobs in the batch.
  2. Created a fresh empty options dict per call that was never read back, so any option mutations (priority, queue, delay, …) were silently discarded.
  3. Bare except: pass swallowed middleware exceptions with no logging.

The fix restructures the dispatch to match single-enqueue's contract: per-job options dict initialised from the call-site arguments, middleware runs before enqueue_batch so mutations can take effect, mutated values are read back into the per-job lists.

What changed

  • py_src/taskito/app.pyenqueue_many() rewritten to:
    • Build a per-job options dict from the call-site arguments (uniform priority/queue/etc. broadcast across the batch; per-job lists threaded through).
    • Run on_enqueue middleware against each per-job dict before enqueue_batch, replacing the silent except: pass with logger.exception("middleware on_enqueue() error").
    • Read the mutated dict back into per-job lists for the Rust call.
  • crates/taskito-python/src/py_queue/mod.rsenqueue_batch signature widened from Option<Vec<i32>> to Option<Vec<Option<i32>>> for priorities / max_retries_list / timeouts. Matches the existing pattern already used for delay_seconds_list / metadata_list / expires_list / result_ttl_list. Allows middleware to set per-job priority while letting other jobs fall back to queue defaults.
  • py_src/taskito/_taskito.pyi — type stub follows the Rust signature change.

Tests

Three new regression tests in tests/python/test_batch.py:

  • test_enqueue_many_invokes_on_enqueue_per_job — middleware sees each job's own args/kwargs.
  • test_enqueue_many_applies_option_mutations — middleware mutating options["priority"] per-job propagates to the actual jobs.
  • test_enqueue_many_logs_middleware_exceptions — exceptions from misbehaving middleware are logged via logger.exception and don't block the enqueue.

Test plan

  • cargo test --workspace — 89 pass
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • cargo check --features postgres clean
  • cargo check --features redis clean
  • uv run python -m pytest tests/python/ — 488 pass, 9 skipped (was 485, +3 new)
  • uv run ruff check py_src/ tests/ clean
  • uv run mypy py_src/taskito/ clean
  • CI green

`enqueue_many` had three bugs in the on_enqueue dispatch:

1. Always passed `args_list[0]` and `kw_list[0]` to every middleware
   call, so middleware could not distinguish jobs in the batch.
2. Created a fresh empty options dict per call that was never read back,
   so any option mutations (priority, queue, delay, ...) were silently
   discarded.
3. Bare `except: pass` swallowed middleware exceptions with no logging.

Restructure the dispatch to mirror single-enqueue: build a per-job
options dict initialised from the call-site arguments, run middleware
*before* `enqueue_batch` so mutations can take effect, then read the
mutated options back into the per-job lists. Replace the silent except
with `logger.exception(...)` so misbehaving middleware is observable.

To support per-job priority/retries/timeout mutations the Rust
`enqueue_batch` signature is widened from `Option<Vec<i32>>` to
`Option<Vec<Option<i32>>>` (matching the existing pattern for
`delay_seconds_list`, `metadata_list`, etc.); type stub follows.

Three regression tests cover the per-job args, the option mutation
propagation, and the logged-exception path.
@pratyush618 pratyush618 merged commit 2fce5cf into master May 2, 2026
19 checks passed
@pratyush618 pratyush618 deleted the fix/enqueue-many-middleware branch May 2, 2026 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant