Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,59 @@ and this project adheres to
Work in progress for the next release. Add entries here as
changes land, not at tag time.

## [0.11.0] - 2026-05-08

### Added

- **`attune-author regenerate --batch`** — opt-in batch path
that submits all polish requests for stale features as one
Anthropic Message Batches API call (~50% cost vs per-call
pricing). Detaches; results are spliced back into the help
corpus by a follow-up `--resume` invocation.
- Companion flags: `--resume` (poll + splice + write),
`--status` (one-shot query, supports `--json`),
`--cancel` (call SDK cancel + remove state file),
`--force` (with `--batch`: overwrite an existing pending
state file).
- Argparse mutex enforces "exactly one batch-mode flag at a
time"; `--force` and `--json` are modifiers.
- Bare `regenerate` (no batch flags) prints a one-liner hint
when a state file exists; never auto-resumes.
- **`attune_author.maintenance_batch`** — new module containing
`submit_maintenance_batch`, `resume_maintenance_batch`,
`status_maintenance_batch`, `cancel_maintenance_batch` plus
the `BatchState` schema and `.help/.batch-state.json`
read/write/delete helpers. Includes 29-day stale-state
detection (Anthropic's batch retention window).
- **`attune_author.doc_gen._anthropic_batch`** — SDK wrapper
module exposing `submit_batch` / `poll_batch` plus typed
`BatchPolishRequest` / `BatchPolishResult` records and an
adaptive `timeout_secs` helper. SIGINT during polling cancels
the batch and re-raises.
- **`attune_author.polish.build_polish_prompt`** — extracted
prompt-building helper. Both the synchronous path and the
batch path call it so the wire-level prompts are byte-identical.
- **`attune_author.generator.prepare_polish_phase`** /
**`apply_polish_results`** — extracted Phase 1 (render) and
Phase 3 (write) helpers. The synchronous
`generate_feature_templates` now calls them; the batch path
uses them too. Single source of truth for rendering.
- **Docs**: `docs/regenerate-batch.md` covers when to use which
path, fallback behavior, env-var overrides, cancel semantics,
and stale-state recovery.
- Live integration test at `tests/integration/test_batch_live.py`,
gated on both `ANTHROPIC_API_KEY` and `RUN_LIVE_BATCH=1`. Tagged
`@pytest.mark.live`. Default-skipped; ~$0.02 per run.

### Changed

- **`generate_feature_templates`** internal layout: refactored
to call the new `prepare_polish_phase` and `apply_polish_results`
helpers. **No behavior change** for existing callers — same
inputs produce byte-identical on-disk templates. Verified by
the existing 152 polish tests + the new parity test in
`tests/test_maintenance_batch.py`.

## [0.10.0] - 2026-05-08

### Added
Expand Down
158 changes: 158 additions & 0 deletions docs/regenerate-batch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
# `attune-author regenerate --batch`

Cost-saving alternative to the synchronous regen path. Submits all
polish requests for stale features as a single Anthropic Message
Batches API call (~50% cost savings) and detaches; you splice the
results back into your help corpus with a follow-up `--resume`
invocation.

## TL;DR

```sh
# kick off the batch (returns immediately)
attune-author regenerate --batch
# go to lunch ...
# when ready, splice results
attune-author regenerate --resume
```

## When to use which path

| Path | Cost | Wall-clock latency | Ergonomics |
|----------------------------|------------|-------------------|-------------------------------------|
| Default (synchronous) | full price | seconds-to-minutes | terminal blocks; one command |
| `--batch` (then `--resume`)| ~50% price | minutes-to-hours | terminal returns immediately twice |

Default is right for: small ad-hoc regen (1–5 stale features), CI
that needs a clean exit code in one step, or when you want
sub-minute end-to-end latency.

`--batch` is right for: bulk regen (10+ stale features), nightly
cron jobs where two scheduled invocations is fine, or any time
the cost savings outweighs a few minutes of wait.

## How it works

The `--batch` invocation:

1. Resolves stale features from the staleness report.
2. Renders all templates for those features (Phase 1 of the
normal generator pipeline).
3. Builds polish prompts for every (feature, depth) pair using
the **same** `build_polish_prompt` helper the synchronous
path uses, so the prompts go out byte-identical.
4. Posts a single Anthropic batch (`messages.batches.create`).
5. Writes a small state file at `.help/.batch-state.json`
containing the batch ID and a per-request manifest.
6. Prints a copy-paste-ready `--resume` hint and exits.

The `--resume` invocation:

1. Loads the state file.
2. Polls Anthropic until the batch terminates (or the adaptive
timeout fires).
3. Splices polished text back into per-feature
`GenerationResult` records.
4. Writes the templates exactly the way the synchronous path
does (Phase 3 of the normal generator pipeline).
5. Deletes the state file (on terminal completion) or keeps it
(on poll-loop timeout, so the next `--resume` picks up).

## Resume ergonomics

A few specific affordances make `--resume` painless:

- **No batch ID required.** `--resume` reads the state file
automatically.
- **Bare `regenerate` prints a hint.** If you accidentally run
the synchronous path while a batch is pending, the CLI tells
you about the pending batch (without auto-resuming).
- **Resume-after-timeout works.** If polling hit our 30-min cap
but Anthropic is still working, the state file is **kept** and
you can run `--resume` again later. No work lost.
- **`--status` is one-shot.** Want to know if your batch is done
without committing to a wait? `attune-author regenerate --status`.
Add `--json` for cron parsing.
- **`--cancel` is one-shot.** Changed your mind, or batch is
wedged? `attune-author regenerate --cancel`.

## Per-request failure isolation

If one (feature, depth) request errors inside the batch (rate
limit, content policy, model error), the **whole feature** is
marked as failed in `result.failed`. Other features in the same
batch are still written to disk normally. This mirrors the
synchronous path's per-feature failure isolation.

## Stale-state recovery

Anthropic retains batch results for 29 days. If you somehow leave
a state file around longer than that, `--resume` surfaces a clean
error pointing at the cleanup path:

```
error: batch submitted 2026-04-01T12:00:00+00:00 is older than
Anthropic's 29-day retention window; delete
.help/.batch-state.json and rerun --batch
```

## Multi-batch guard

If you run `--batch` while a state file already exists, you get a
clear refusal:

```
error: pending batch already submitted; state file at
.help/.batch-state.json. Run --resume, --cancel, or pass
--force to overwrite.
```

`--force` overwrites the state file and starts a new batch. (The
old Anthropic batch continues to run on its side; you can let it
finish or cancel via the dashboard.)

## Environment variables

| Variable | Default | Purpose |
|--------------------------------|---------|---------------------------------------------|
| `ATTUNE_BATCH_POLL_SECS` | 30 | Poll interval for `--resume`. |
| `ATTUNE_BATCH_TIMEOUT_SECS` | adaptive | Override the poll-loop ceiling. |
| `ANTHROPIC_API_KEY` | — | Required for any LLM call. |

The adaptive timeout default scales with batch size:
`min(30min, max(5min, 60·N/20))` — 5min minimum (handles
cold-start variance) plus 1min per 20 requests, capped at
30min.

## Ctrl+C semantics

During `--resume` polling, SIGINT calls Anthropic's
`messages.batches.cancel` and re-raises so the CLI can clean up.
Anthropic doesn't refund work already in progress, but it stops
new work and lets you walk away without lingering charges.

## Cost model

The Batches API charges ~50% of the per-token prices of the
synchronous Messages API. There's no per-batch overhead; cost is
purely a function of total tokens. Batch jobs typically complete
within minutes for our N (~10–50 polish prompts), well inside
Anthropic's 24h SLA.

## Live integration test

`tests/integration/test_batch_live.py` covers end-to-end
submit→poll→splice with two real polish requests. Skipped by
default; opt-in via:

```sh
ANTHROPIC_API_KEY=sk-ant-... RUN_LIVE_BATCH=1 \
pytest tests/integration/test_batch_live.py -m live
```

Cost: ~$0.02 per run.

## See also

- Spec: `specs/author-batch-maintain/{requirements,design,tasks}.md`
- Anthropic docs: <https://docs.anthropic.com/en/docs/build-with-claude/batch-processing>
5 changes: 4 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "attune-author"
version = "0.10.0"
version = "0.11.0"
description = "Documentation authoring and maintenance for the attune ecosystem — generate, maintain, and validate help content with AI assistance."
readme = {file = "README.md", content-type = "text/markdown"}
requires-python = ">=3.10"
Expand Down Expand Up @@ -102,6 +102,9 @@ select = ["E", "F", "W", "I", "UP", "BLE"]

[tool.pytest.ini_options]
testpaths = ["tests"]
markers = [
"live: opt-in tests that hit the live Anthropic API. Skipped by default; require ANTHROPIC_API_KEY and any test-specific env flags.",
]

[tool.coverage.run]
source = ["attune_author"]
2 changes: 1 addition & 1 deletion src/attune_author/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
attune-help (reader) and attune-ai (full dev workflows).
"""

__version__ = "0.10.0"
__version__ = "0.11.0"

from attune_author.manifest import Feature, Manifest, load_manifest
from attune_author.staleness import StalenessReport, check_staleness, compute_source_hash
Expand Down
Loading
Loading