From 17df90ee884c8c78eec0daaa28c8e45aabea2bc5 Mon Sep 17 00:00:00 2001 From: Luke Oliff Date: Thu, 5 Mar 2026 12:41:33 +0000 Subject: [PATCH 1/2] chore: add Fern regen workflow docs and skills Add AGENTS.md documenting the SDK regeneration workflow, CLAUDE.md stub, and Claude Code skills for prepare-regen and review-regen. --- .claude/skills/prepare-regen.md | 17 +++++++++ .claude/skills/review-regen.md | 21 +++++++++++ AGENTS.md | 65 +++++++++++++++++++++++++++++++++ CLAUDE.md | 1 + 4 files changed, 104 insertions(+) create mode 100644 .claude/skills/prepare-regen.md create mode 100644 .claude/skills/review-regen.md create mode 100644 AGENTS.md create mode 100644 CLAUDE.md diff --git a/.claude/skills/prepare-regen.md b/.claude/skills/prepare-regen.md new file mode 100644 index 00000000..80cab939 --- /dev/null +++ b/.claude/skills/prepare-regen.md @@ -0,0 +1,17 @@ +# Prepare Repo for Fern SDK Regeneration + +Triggers: prepare regen, prepare for regeneration, prep for fern, new sdk gen, prepare sdk gen + +## Steps + +Read AGENTS.md for full context on the regeneration workflow. + +1. Fetch latest `main` and create a new branch: `lo/sdk-gen-` (use today's date). +2. Push the branch to origin. +3. Create an empty commit (`chore: initialize SDK regeneration branch`) if needed, then create a PR titled `chore: SDK regeneration ` with body `## Summary\n- Fern SDK regeneration`. +4. Read `.fernignore` and identify all frozen generated files (Client & Socket Clients, Type Fixes, Redact type, Listen Clients, Tests & WireMock). Do NOT unfreeze the "always frozen" files listed in AGENTS.md. +5. Back up each identified file as `.bak` alongside the original. +6. Remove their entries and associated comments from `.fernignore`. +7. Stage the updated `.fernignore` and all `.bak` files. +8. Commit as `chore: unfreeze files pending regen` and push. +9. Report the PR URL and confirm the branch is ready for generator output. diff --git a/.claude/skills/review-regen.md b/.claude/skills/review-regen.md new file mode 100644 index 00000000..b563de69 --- /dev/null +++ b/.claude/skills/review-regen.md @@ -0,0 +1,21 @@ +# Review Fern SDK Regeneration + +Triggers: review regen, review regeneration, diff regen, compare regen, post regen review + +## Steps + +Read AGENTS.md for full context on the regeneration workflow. + +1. Find all `.bak` files in the repo — these are the pre-regen manually-patched versions. +2. For each `.bak` file, diff it against the corresponding newly generated file to show what changed. +3. Identify which manual patches are still needed (the `.bak` has changes the generator doesn't produce). +4. Present a summary of findings to the user, grouped by category: + - Patches no longer needed (generator now handles it) + - Patches still needed (must be re-applied) + - New changes from the generator worth noting +5. Wait for user direction on which patches to re-apply. +6. Re-apply confirmed patches to the generated files. +7. Re-add entries to `.fernignore` for files that still carry manual patches. +8. Delete all `.bak` files. +9. Run `ruff check --fix`, `mypy --ignore-missing-imports src/deepgram/`, and `pytest` to verify. +10. Commit as `chore: re-apply manual patches after regen` and push. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..c2fe6127 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,65 @@ +# Agents + +## Fern SDK Regeneration + +### Overview + +This SDK is generated by [Fern](https://buildwithfern.com/). Most files under `src/deepgram/` are auto-generated and should not be edited directly. Some files have manual patches (custom auth, type fixes, WebSocket defaults, etc.) and are listed in `.fernignore` to prevent the generator from overwriting them. + +When a new Fern generator release is available, we prepare the repo so the generator can overwrite previously-frozen files, then re-apply manual patches after reviewing the diff. + +### Files that may be frozen in `.fernignore` + +These are the files that typically carry manual patches and get frozen between regenerations: + +**Client & Socket Clients:** +- `src/deepgram/client.py` — Bearer token auth + session ID injection +- `src/deepgram/speak/v1/socket_client.py` — optional message param defaults, broad exception catch +- `src/deepgram/listen/v1/socket_client.py` — same as above + `construct_type` for unknown WS messages +- `src/deepgram/listen/v2/socket_client.py` — same as above +- `src/deepgram/agent/v1/socket_client.py` — same as above + `_sanitize_numeric_types` + +**Type Fixes (Fern generates float where API returns int):** +- `src/deepgram/types/listen_v1response_results_utterances_item.py` +- `src/deepgram/types/listen_v1response_results_utterances_item_words_item.py` +- `src/deepgram/types/listen_v1response_results_channels_item_alternatives_item_paragraphs_paragraphs_item.py` + +**Redact type (Union[str, Sequence[str]] support):** +- `src/deepgram/types/listen_v1redact.py` + +**Listen Clients (array param support):** +- `src/deepgram/listen/v1/client.py` +- `src/deepgram/listen/v2/client.py` + +**Tests & WireMock:** +- `tests/wire/test_listen_v1_media.py` — manual fix for `transcribe_file()` bytes param +- `wiremock/wiremock-mappings.json` — removed duplicate stub causing non-deterministic matching + +**Always frozen (never unfreeze for regen):** +- `tests/custom/test_text_builder.py`, `tests/custom/test_transport.py` — hand-written tests +- `tests/manual/` — manual standalone tests +- `README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`, `reference.md` — docs +- `src/deepgram/helpers/` — TextBuilder helpers +- `src/deepgram/transport_interface.py`, `src/deepgram/transport.py`, `src/deepgram/transports/` — custom transport support +- `.github/`, `docs/`, `examples/` — folders + +### Prepare repo for regeneration + +1. **Create a new branch** off `main` named `lo/sdk-gen-`. +2. **Push the branch** and create a PR titled `chore: SDK regeneration ` with an empty commit if needed. +3. **Identify frozen generated files** in `.fernignore` (the files listed above under Client, Type Fixes, Redact, Listen Clients, Tests & WireMock — NOT the "always frozen" files). +4. **Back up each file** as `.bak` alongside the original. +5. **Remove their entries** (and associated comments) from `.fernignore`. +6. **Commit** as `chore: unfreeze files pending regen` and push. +7. The branch is now ready for the Fern generator to push changes. + +### After regeneration + +The `.bak` files are our manually-patched versions. The newly generated files replace them. By comparing the two, we can see what the generator now produces vs what we had patched. + +1. **Diff each `.bak` file against the new generated version** (e.g., `diff src/deepgram/client.py.bak src/deepgram/client.py`) to understand what changed and whether our patches are still needed. +2. **Re-apply any patches** that are still necessary to the newly generated files. +3. **Re-add entries to `.fernignore`** for any files that still need manual patches. +4. **Delete `.bak` files** once review is complete. +5. **Run tests** (`pytest`) and linting (`ruff check`, `mypy`) to verify. +6. **Commit** as `chore: re-apply manual patches after regen` and push. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..43c994c2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md From d084961398b2f5c7a07a0a4155bcd2681f3a7ec5 Mon Sep 17 00:00:00 2001 From: Luke Oliff Date: Thu, 5 Mar 2026 12:48:32 +0000 Subject: [PATCH 2/2] chore: add regen workflow docs with freeze classification rules Add AGENTS.md with holistic rules for classifying .fernignore entries as permanently or temporarily frozen. Add Claude Code skills for prepare-regen and review-regen. Add agent files to .fernignore. --- .claude/skills/prepare-regen.md | 19 ++++---- .claude/skills/review-regen.md | 13 ++--- .fernignore | 5 ++ AGENTS.md | 85 +++++++++++++++++++-------------- 4 files changed, 72 insertions(+), 50 deletions(-) diff --git a/.claude/skills/prepare-regen.md b/.claude/skills/prepare-regen.md index 80cab939..bd5c0569 100644 --- a/.claude/skills/prepare-regen.md +++ b/.claude/skills/prepare-regen.md @@ -4,14 +4,17 @@ Triggers: prepare regen, prepare for regeneration, prep for fern, new sdk gen, p ## Steps -Read AGENTS.md for full context on the regeneration workflow. +Read AGENTS.md for full context on the regeneration workflow and freeze classification rules. 1. Fetch latest `main` and create a new branch: `lo/sdk-gen-` (use today's date). 2. Push the branch to origin. -3. Create an empty commit (`chore: initialize SDK regeneration branch`) if needed, then create a PR titled `chore: SDK regeneration ` with body `## Summary\n- Fern SDK regeneration`. -4. Read `.fernignore` and identify all frozen generated files (Client & Socket Clients, Type Fixes, Redact type, Listen Clients, Tests & WireMock). Do NOT unfreeze the "always frozen" files listed in AGENTS.md. -5. Back up each identified file as `.bak` alongside the original. -6. Remove their entries and associated comments from `.fernignore`. -7. Stage the updated `.fernignore` and all `.bak` files. -8. Commit as `chore: unfreeze files pending regen` and push. -9. Report the PR URL and confirm the branch is ready for generator output. +3. Create an empty commit (`chore: initialize SDK regeneration branch`) if needed, then create a PR titled `chore: SDK regeneration `. +4. Read `.fernignore` and classify each entry using the rules in AGENTS.md: + - **Permanently frozen**: entirely hand-written, no Fern equivalent. NEVER touch these. + - **Temporarily frozen**: Fern-generated with manual patches. These get swapped. +5. For each **temporarily frozen** file only: + - Copy the file to `.bak` alongside the original. + - In `.fernignore`, replace the original path with the `.bak` path (protects our patch, lets Fern overwrite the original). +6. Stage the updated `.fernignore` and all `.bak` files. +7. Commit as `chore: unfreeze files pending regen` and push. +8. Report the PR URL and confirm the branch is ready for generator output. diff --git a/.claude/skills/review-regen.md b/.claude/skills/review-regen.md index b563de69..8511bca2 100644 --- a/.claude/skills/review-regen.md +++ b/.claude/skills/review-regen.md @@ -4,9 +4,9 @@ Triggers: review regen, review regeneration, diff regen, compare regen, post reg ## Steps -Read AGENTS.md for full context on the regeneration workflow. +Read AGENTS.md for full context on the regeneration workflow and freeze classification rules. -1. Find all `.bak` files in the repo — these are the pre-regen manually-patched versions. +1. Find all `.bak` files in the repo — these are our pre-regen manually-patched versions (protected by `.fernignore`). 2. For each `.bak` file, diff it against the corresponding newly generated file to show what changed. 3. Identify which manual patches are still needed (the `.bak` has changes the generator doesn't produce). 4. Present a summary of findings to the user, grouped by category: @@ -15,7 +15,8 @@ Read AGENTS.md for full context on the regeneration workflow. - New changes from the generator worth noting 5. Wait for user direction on which patches to re-apply. 6. Re-apply confirmed patches to the generated files. -7. Re-add entries to `.fernignore` for files that still carry manual patches. -8. Delete all `.bak` files. -9. Run `ruff check --fix`, `mypy --ignore-missing-imports src/deepgram/`, and `pytest` to verify. -10. Commit as `chore: re-apply manual patches after regen` and push. +7. In `.fernignore`, replace each `.bak` path back to the original path for files that still need patches. +8. Remove `.fernignore` entries entirely for files where patches are no longer needed. +9. Delete all `.bak` files. +10. Run `ruff check --fix`, `mypy --ignore-missing-imports src/deepgram/`, and `pytest` to verify. +11. Commit as `chore: re-apply manual patches after regen` and push. diff --git a/.fernignore b/.fernignore index cbea1734..1086ff6d 100644 --- a/.fernignore +++ b/.fernignore @@ -67,6 +67,11 @@ src/deepgram/transport_interface.py src/deepgram/transport.py src/deepgram/transports +# Claude Code agent files +CLAUDE.md +AGENTS.md +.claude + # Folders to ignore .github docs diff --git a/AGENTS.md b/AGENTS.md index c2fe6127..91040644 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,62 +4,75 @@ ### Overview -This SDK is generated by [Fern](https://buildwithfern.com/). Most files under `src/deepgram/` are auto-generated and should not be edited directly. Some files have manual patches (custom auth, type fixes, WebSocket defaults, etc.) and are listed in `.fernignore` to prevent the generator from overwriting them. +This SDK is generated by [Fern](https://buildwithfern.com/). Most files under `src/deepgram/` are auto-generated and should not be edited directly. Some files have manual patches and are listed in `.fernignore` to prevent the generator from overwriting them. When a new Fern generator release is available, we prepare the repo so the generator can overwrite previously-frozen files, then re-apply manual patches after reviewing the diff. -### Files that may be frozen in `.fernignore` +### Freeze classification rules -These are the files that typically carry manual patches and get frozen between regenerations: +Every entry in `.fernignore` falls into one of two categories. The **comment above each entry** in `.fernignore` indicates which category it belongs to, but when in doubt, apply these rules: -**Client & Socket Clients:** -- `src/deepgram/client.py` — Bearer token auth + session ID injection -- `src/deepgram/speak/v1/socket_client.py` — optional message param defaults, broad exception catch -- `src/deepgram/listen/v1/socket_client.py` — same as above + `construct_type` for unknown WS messages -- `src/deepgram/listen/v2/socket_client.py` — same as above -- `src/deepgram/agent/v1/socket_client.py` — same as above + `_sanitize_numeric_types` - -**Type Fixes (Fern generates float where API returns int):** -- `src/deepgram/types/listen_v1response_results_utterances_item.py` -- `src/deepgram/types/listen_v1response_results_utterances_item_words_item.py` -- `src/deepgram/types/listen_v1response_results_channels_item_alternatives_item_paragraphs_paragraphs_item.py` +#### Never unfreeze (permanently frozen) -**Redact type (Union[str, Sequence[str]] support):** -- `src/deepgram/types/listen_v1redact.py` +These files are **entirely hand-written** — they have no Fern-generated counterpart. The generator would delete or replace them with something unrelated. They must stay in `.fernignore` at all times. -**Listen Clients (array param support):** -- `src/deepgram/listen/v1/client.py` -- `src/deepgram/listen/v2/client.py` +How to identify: +- The file was **created by us**, not by Fern (e.g., `src/deepgram/client.py`, custom tests, helpers, transport layer) +- The file is a **doc, config, or folder** we maintain independently (README, CHANGELOG, .github, examples, etc.) +- The file lives **outside `src/deepgram/`** in a hand-maintained location (e.g., `.claude/`, `docs/`) -**Tests & WireMock:** -- `tests/wire/test_listen_v1_media.py` — manual fix for `transcribe_file()` bytes param -- `wiremock/wiremock-mappings.json` — removed duplicate stub causing non-deterministic matching - -**Always frozen (never unfreeze for regen):** +Current permanently frozen files: +- `src/deepgram/client.py` — entirely custom (Bearer auth, session ID); no Fern equivalent +- `src/deepgram/helpers/` — hand-written TextBuilder helpers +- `src/deepgram/transport_interface.py`, `src/deepgram/transport.py`, `src/deepgram/transports/` — custom transport layer - `tests/custom/test_text_builder.py`, `tests/custom/test_transport.py` — hand-written tests - `tests/manual/` — manual standalone tests - `README.md`, `CHANGELOG.md`, `CONTRIBUTING.md`, `reference.md` — docs -- `src/deepgram/helpers/` — TextBuilder helpers -- `src/deepgram/transport_interface.py`, `src/deepgram/transport.py`, `src/deepgram/transports/` — custom transport support +- `CLAUDE.md`, `AGENTS.md`, `.claude/` — agent files - `.github/`, `docs/`, `examples/` — folders +#### Unfreeze for regen (temporarily frozen) + +These files are **Fern-generated but carry manual patches** to fix issues in the generator output. We freeze them to protect our patches between regenerations, but unfreeze them before a regen so we can compare the new output against our patches. + +How to identify: +- The file **exists in Fern's output** — if you removed it from `.fernignore` and ran the generator, Fern would produce a version of it +- Our version is a **modified copy** of what Fern generates (e.g., changed `float` to `int`, added optional defaults, broadened a Union type) + +Current temporarily frozen files: +- `src/deepgram/speak/v1/socket_client.py` — optional message param defaults, broad exception catch +- `src/deepgram/listen/v1/socket_client.py` — same + `construct_type` for unknown WS messages +- `src/deepgram/listen/v2/socket_client.py` — same +- `src/deepgram/agent/v1/socket_client.py` — same + `_sanitize_numeric_types` +- `src/deepgram/types/listen_v1response_results_utterances_item.py` — `float` → `int` fix +- `src/deepgram/types/listen_v1response_results_utterances_item_words_item.py` — `float` → `int` fix +- `src/deepgram/types/listen_v1response_results_channels_item_alternatives_item_paragraphs_paragraphs_item.py` — `float` → `int` fix +- `src/deepgram/types/listen_v1redact.py` — Union[str, Sequence[str]] support +- `src/deepgram/listen/v1/client.py` — Union[str, Sequence[str]] array param support +- `src/deepgram/listen/v2/client.py` — same +- `tests/wire/test_listen_v1_media.py` — `transcribe_file()` bytes param fix +- `wiremock/wiremock-mappings.json` — removed duplicate stub + ### Prepare repo for regeneration 1. **Create a new branch** off `main` named `lo/sdk-gen-`. -2. **Push the branch** and create a PR titled `chore: SDK regeneration ` with an empty commit if needed. -3. **Identify frozen generated files** in `.fernignore` (the files listed above under Client, Type Fixes, Redact, Listen Clients, Tests & WireMock — NOT the "always frozen" files). -4. **Back up each file** as `.bak` alongside the original. -5. **Remove their entries** (and associated comments) from `.fernignore`. +2. **Push the branch** and create a PR titled `chore: SDK regeneration ` (empty commit if needed). +3. **Read `.fernignore`** and classify each entry using the rules above. +4. **For each temporarily frozen file only:** + - Copy the file to `.bak` alongside the original. + - In `.fernignore`, **replace the original path with the `.bak` path**. This protects our patched version from the generator while allowing Fern to overwrite the original. +5. **Never touch permanently frozen entries.** Leave them in `.fernignore` as-is. 6. **Commit** as `chore: unfreeze files pending regen` and push. 7. The branch is now ready for the Fern generator to push changes. ### After regeneration -The `.bak` files are our manually-patched versions. The newly generated files replace them. By comparing the two, we can see what the generator now produces vs what we had patched. +The `.bak` files are our manually-patched versions (protected by `.fernignore`). The original paths now contain the freshly generated versions. By comparing the two, we can see what the generator now produces vs what we had patched. -1. **Diff each `.bak` file against the new generated version** (e.g., `diff src/deepgram/client.py.bak src/deepgram/client.py`) to understand what changed and whether our patches are still needed. +1. **Diff each `.bak` file against the new generated version** to understand what changed and whether our patches are still needed. 2. **Re-apply any patches** that are still necessary to the newly generated files. -3. **Re-add entries to `.fernignore`** for any files that still need manual patches. -4. **Delete `.bak` files** once review is complete. -5. **Run tests** (`pytest`) and linting (`ruff check`, `mypy`) to verify. -6. **Commit** as `chore: re-apply manual patches after regen` and push. +3. **In `.fernignore`, replace each `.bak` path back to the original path** for any files that still need manual patches. +4. **Remove `.fernignore` entries entirely** for any files where the generator now produces correct output (patches no longer needed). +5. **Delete all `.bak` files** once review is complete. +6. **Run tests** (`pytest`) and linting (`ruff check`, `mypy`) to verify. +7. **Commit** as `chore: re-apply manual patches after regen` and push.