Add Batch Processing Patterns category with 4 new patterns#6
Open
joshmsmith wants to merge 3 commits into
Open
Add Batch Processing Patterns category with 4 new patterns#6joshmsmith wants to merge 3 commits into
joshmsmith wants to merge 3 commits into
Conversation
- batch iterator - fanout - mapreduce tree - sliding window
## Summary
Adds a new **Batch Processing Patterns** category to the catalog with an overview page and four fully-documented patterns, each with runnable sandbox samples in TypeScript, Python, Go, and Java.
## New patterns
| Pattern | Description |
|---|---|
| **Overview** (`batch-processing-patterns.md`) | Decision table and tile grid helping readers pick the right pattern based on record set size, parallelism model, and rate-control needs |
| **Fan-Out with Child Workflows** (`fanout-child-workflows.md`) | Splits a record set into fixed-size chunks and assigns each to an independent child Workflow. Best for record sets up to ~4M items. |
| **Batch Iterator** (`batch-iterator.md`) | Processes an unbounded record set one page at a time using Continue-As-New, with configurable per-page rate control. |
| **Sliding Window** (`sliding-window.md`) | Maintains a bounded window of concurrently running child Workflows, launching a new one each time one completes. |
| **MapReduce Tree** (`mapreduce-tree.md`) | Recursively splits the record set into halves until each leaf processes a small chunk, then aggregates results back up the tree. Maximum parallelism, no rate control. |
## Sandbox samples
Each pattern ships runnable code in all four SDKs under `sandbox-runner/patterns/{batch-iterator,sliding-window,fanout-child-workflows,mapreduce-tree}/`. Each directory includes `pattern.json`, per-language source files, Go build-cache warmup stubs, and Java Maven warmup stubs.
## Sidebar
Added a "Batch Processing Patterns" section to `docs/.vitepress/config.mts`.
## Out of scope
The event-accumulator skeleton (`sandbox-runner/patterns/event-accumulator/`) is not included and will be completed separately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new Batch Processing Patterns category to the catalog with an overview page and four fully-documented patterns, each with runnable sandbox samples in TypeScript, Python, Go, and Java.
New patterns
batch-processing-patterns.md)fanout-child-workflows.md)batch-iterator.md)sliding-window.md)mapreduce-tree.md)Sandbox samples
Each pattern ships runnable code in all four SDKs under
sandbox-runner/patterns/{batch-iterator,sliding-window,fanout-child-workflows,mapreduce-tree}/. Each directory includespattern.json, per-language source files, Go build-cache warmup stubs, and Java Maven warmup stubs.Sidebar
Added a "Batch Processing Patterns" section to
docs/.vitepress/config.mts.Out of scope
The event-accumulator skeleton (
sandbox-runner/patterns/event-accumulator/) is not included and will be completed separately.