Skip to content

Commit 1d93b5c

Browse files
lundybernardlundybernard
committed
docs: add MyST migration planning docs and ADRs
Adds the full planning record for the Hugo → MyST-MD migration to docs/decisions/0001-myst-migration/: - REQUIREMENTS.md — what the migration must achieve and why - PLAN.md — commit-by-commit 8-phase implementation sequence - 0001–0007 ADR files — one decision per file, all Status: Proposed - README.md — index of the above ADR format is used so the decision rationale survives as a reviewable archive. All ADRs are Proposed pending maintainer review; the plan and requirements are the working basis for the migration work to follow.
1 parent ca1bf2d commit 1d93b5c

11 files changed

Lines changed: 1019 additions & 0 deletions

‎docs/decisions/0001-myst-migration/0001-migrate-to-mystmd.md‎

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# ADR 0001 — Migrate build system from Hugo to MyST-MD
2+
3+
Date: 2026-05-11
4+
Status: Proposed — **open question for maintainers (see below)**
5+
Branch: lb/myst-migration
6+
Issue: scientific-python/scientific-python.org#846
7+
8+
## Context
9+
10+
`learn.scientific-python.org` builds with Hugo via `make html` and deploys via
11+
Netlify (auto-deploy on push to `main`). We want to convert the content files
12+
to MyST syntax.
13+
14+
Three realistic MyST toolchain options exist. They are not equivalent:
15+
`jupyter-book` is a higher-level tool built on top of `mystmd`; `mystmd` is the
16+
underlying engine and is available as both a Node package (npm) and a Python
17+
package (pip/conda) that bundles Node internally.
18+
19+
## Decision (proposed)
20+
21+
Replace Hugo with **`mystmd` Python package** (`pip install mystmd`,
22+
`myst build --html`) as the build tool.
23+
24+
## Options considered
25+
26+
| Option | Pros | Cons |
27+
| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
28+
| **Stay on Hugo** | Theme parity with sibling SP sites; no content changes needed | Hugo is not Python; MyST content conversion still desirable long-term |
29+
| **jupyter-book 2.x** | Pure Python (`pip install jupyter-book`); SP ecosystem familiar with JB; conda-forge package; handles notebook execution natively | Wraps `mystmd` under the hood — extra abstraction; config format (`_config.yml`, `_toc.yml`) is not portable to plain `myst.yml` if JB is dropped later; JB 2.x released late 2024 — docs and community experience thin; feature lag vs direct `mystmd` |
30+
| **mystmd — Node CLI** (`npm install mystmd`) | Native runtime; latest npm releases immediately; same `myst.yml` config; active ExecutableBooks development | Requires Node.js in every build environment (Netlify, RTD, CI); unfamiliar to Python contributors |
31+
| **mystmd — Python package** (`pip install mystmd`) | Python-centric install (pip/conda-forge); bundles Node internally — no separate Node needed; same `myst.yml` config as Node CLI (no extra abstraction layer); proven by `tools.scientific-python.org` PR #81; works on all considered deploy platforms | Node bundled internally — slightly opaque; PyPI/conda releases may lag npm by 1–7 days |
32+
33+
## Rationale for proposed decision
34+
35+
**Why not jupyter-book:** `learn.scientific-python.org` contains no Jupyter
36+
notebooks; JB's primary value (notebook execution, Sphinx integration) does not
37+
apply here. JB 2.x uses `mystmd` as its build engine, so the team would get
38+
`mystmd` indirectly with an extra config layer on top. The JB config format
39+
(`_config.yml`, `_toc.yml`) is not portable — if JB were dropped later, the
40+
config would need to be rewritten to `myst.yml` from scratch.
41+
42+
**Why not the Node CLI:** Requires Node.js in every build environment. SP
43+
contributors and maintainers work in Python environments; npm is unfamiliar and
44+
adds friction for new contributors. The Python package provides identical
45+
functionality without any Node setup.
46+
47+
**Why the Python package:** Fits SP's Python-centric workflow; `conda install
48+
-c conda-forge mystmd` works for conda users. No Node.js needed in Netlify,
49+
RTD, or GitHub Actions (Node is bundled inside the package). The `myst.yml`
50+
config is identical to the Node CLI — switching delivery method later is a
51+
one-line change.
52+
53+
## Open question for maintainers
54+
55+
> **Which MyST toolchain should `learn.scientific-python.org` adopt?**
56+
>
57+
> A. `mystmd` Python package — proposed above (`pip install mystmd`)
58+
> B. `jupyter-book 2.x` — if the team prefers a unified JB-based approach
59+
> C. `mystmd` Node CLI — if the team prefers the native Node runtime
60+
>
61+
> This is the foundational decision for the migration. All downstream ADRs
62+
> (0002–0007) assume option A. If maintainers choose B, the `myst.yml` /
63+
> `_toc.yml` structure and config format change significantly. Option C
64+
> requires Node.js toolchain setup in `netlify.toml` and CI.
65+
66+
## Installation
67+
68+
For Netlify builds and RTD: `pip install mystmd` (no Node configuration needed
69+
in `netlify.toml`; see ADR 0007).
70+
71+
Local dev: install via any preferred method (pip, conda, npm — developer's
72+
choice). The repo does not mandate a specific local environment.
73+
74+
## Consequences
75+
76+
- `myst build --html` replaces `make html` (Hugo)
77+
- `pip install mystmd` is the chosen delivery method; no Node toolchain required
78+
in CI or `netlify.toml`
79+
- The `scientific-python-hugo-theme` submodule is removed (see ADR 0003)
80+
- `netlify.toml` is updated to use `mystmd` (see PLAN.md Phase 6)
81+
- Nine content files require shortcode conversion (see ADR 0002)
82+
- Footer/quicklinks are not yet supported in MyST default templates (see ADR 0005)
83+
- Other SP repos remain on Hugo until they choose to migrate (see ADR 0006)

‎docs/decisions/0001-myst-migration/0002-shortcode-mapping.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# ADR 0002 — Hugo shortcode → MyST directive mapping
2+
3+
Date: 2026-05-11
4+
Status: Proposed
5+
Branch: lb/myst-migration
6+
7+
## Context
8+
9+
Nine content files use Hugo shortcodes that MyST does not understand:
10+
11+
| Shortcode type | Files affected (pre-Phase-2 names) |
12+
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
13+
| `{{< grid >}}` / `[[item]]` | `_index.md` (root), `contributors/_index.md`, `documentation/_index.md` |
14+
| `{{< admonition >}}` | `maintainers/_index.md`, `maintainers/interacting-with-new-contributors.md`, `maintainers/managing-conflict.md`, `maintainers/meeting_types.md`, `contributors/first-contribution.md`, `community/onboarding.md` |
15+
16+
## Decision
17+
18+
Convert shortcodes in two **type-batched commits** (one per shortcode type,
19+
not one per file), using the following canonical mappings:
20+
21+
```
22+
{{< grid columns="1 2 2 3" >}} → ::::{grid} 1 2 2 3
23+
:gutter: 2
24+
25+
[[item]] → :::{card} <title>
26+
type = 'card' :link: <link>
27+
title = 'X'
28+
link = 'y' <body>
29+
body = 'z' :::
30+
31+
{{< /grid >}} → ::::
32+
33+
{{< admonition warning >}}… → :::{warning}
34+
{{< /admonition >}} …
35+
:::
36+
37+
{{< admonition note >}}… → :::{note}
38+
{{< /admonition >}} …
39+
:::
40+
```
41+
42+
## Options considered
43+
44+
- **Custom MyST plugin** to interpret Hugo TOML shortcode syntax — high effort,
45+
no value once content is converted.
46+
- **Per-file commits** — nine files but ten shortcode occurrences (one file
47+
has two grid blocks); reviewers re-read the same mapping repeatedly.
48+
- **Type-batched commits** — two diffs total; reviewers evaluate the mapping
49+
pattern once per type.
50+
51+
## Consequences
52+
53+
- Two commits in Phase 3 of the migration (see [PLAN.md](PLAN.md))
54+
- Reviewers can verify correctness by comparing rendered output against the
55+
Hugo-built site for these nine pages
56+
- `grep -rE '\{\{<' content/` returns zero matches after these commits
57+
- `content/index.md` is the canonical style reference going forward

‎docs/decisions/0001-myst-migration/0003-remove-hugo-theme-submodule.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# ADR 0003 — Remove scientific-python-hugo-theme submodule
2+
3+
Date: 2026-05-11
4+
Status: Proposed
5+
Branch: lb/myst-migration
6+
7+
## Context
8+
9+
`themes/scientific-python-hugo-theme` is a git submodule pinned to `v0.21`.
10+
The same theme submodule is also referenced by three sibling repos:
11+
`scientific-python.org`, `blog.scientific-python.org`, and
12+
`tools.scientific-python.org`. Each repo pins the submodule independently in
13+
its own `.gitmodules`; they do not share a checkout.
14+
15+
MyST-MD does not use Hugo themes. Once Hugo is removed from this repo, the
16+
submodule has no consumer here.
17+
18+
## Decision
19+
20+
Remove the submodule from this repo in Phase 7 of the migration.
21+
22+
```bash
23+
git submodule deinit -f themes/scientific-python-hugo-theme
24+
git rm themes/scientific-python-hugo-theme
25+
rm -rf .git/modules/themes/scientific-python-hugo-theme
26+
```
27+
28+
## Options considered
29+
30+
1. **Remove in this PR** — clean cut; no dead code after Hugo is gone.
31+
2. **Keep until all four SP repos migrate** — delays cleanup by months or
32+
quarters; leaves a submodule that nothing in this repo uses.
33+
3. **Vendor a snapshot** — no benefit; MyST doesn't use it.
34+
35+
## Consequences
36+
37+
- The `themes/` directory is deleted from this repo
38+
- The three sibling repos are **unaffected** — they reference the submodule
39+
from their own `.gitmodules` and pin their own SHA
40+
- The upstream `scientific-python-hugo-theme` repo is not affected
41+
- A follow-up PR to the sibling repos removes their copies when they migrate
42+
(see ADR 0006)

‎docs/decisions/0001-myst-migration/0004-defer-cookie-jekyll.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# ADR 0004 — Defer migration of external-content/cookie (Jekyll)
2+
3+
Date: 2026-05-11
4+
Status: Proposed
5+
Branch: lb/myst-migration
6+
7+
## Context
8+
9+
`external-content/cookie` is a Jekyll site (git submodule, pinned to
10+
`2025.10.01`). It is built separately via `make cookie` and its output is
11+
merged into `public/development/` before deploy. It has its own upstream
12+
release cadence and contributors independent of `learn`.
13+
14+
## Decision
15+
16+
Leave `external-content/cookie` unchanged in this PR. File a follow-up issue:
17+
_"MyST: migrate external-content/cookie off Jekyll"_.
18+
19+
The `make cookie`, `make external`, `cookie_ruby_deps`, `cookie_web_prepare`,
20+
and `prepare` Makefile targets are preserved. `make html-all` continues to
21+
build the MyST site then merge the Jekyll output into `public/`.
22+
23+
## Options considered
24+
25+
1. **Keep as-is permanently** — MyST output and Jekyll output coexist forever;
26+
`ghp-import` merges them. Lowest risk but leaves a Jekyll dependency
27+
indefinitely.
28+
2. **Convert cookie to MyST** — large, independent effort; distracts from this
29+
PR and would require its own review.
30+
3. **Drop cookie** — would break the `/development/` path; not acceptable
31+
without a replacement.
32+
4. **Defer with follow-up issue** — keep `make cookie` working now; track
33+
conversion separately so it gets its own focused review.
34+
35+
## Consequences
36+
37+
- `external-content/cookie` submodule remains at `2025.10.01`
38+
- `public/development/` continues to be produced by Jekyll
39+
- The `html-all` Makefile target reconciles MyST's `_build/html/` output
40+
with cookie's `public/development/` via `mkdir -p public && cp -r _build/html/* public/`
41+
before `make external` overlays `/development/`
42+
- A follow-up issue tracks the eventual Jekyll → MyST conversion

‎docs/decisions/0001-myst-migration/0005-defer-footer-quicklinks.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# ADR 0005 — Defer footer and quicklinks to follow-up
2+
3+
Date: 2026-05-11
4+
Status: Proposed
5+
Branch: lb/myst-migration
6+
7+
## Context
8+
9+
Hugo `config.yaml` defines two visual elements that have no direct equivalent
10+
in MyST default templates:
11+
12+
1. **Footer social icons** — GitHub, YouTube, Mastodon, Discourse, Discord
13+
2. **Quicklinks columns** — three columns of site-wide nav links (About,
14+
Maintainers/SPECs, Press kit)
15+
16+
MyST's default HTML template renders a minimal footer with no configurable
17+
social links or quicklinks.
18+
19+
## Decision
20+
21+
Ship this PR with the MyST default footer. File a follow-up issue:
22+
_"MyST: footer + quicklinks parity with Hugo theme"_.
23+
24+
Add a comment block at the bottom of `myst.yml` pointing at the follow-up
25+
issue number so the gap is immediately discoverable.
26+
27+
## Options considered
28+
29+
1. **Custom MyST theme / template override** — achieves full parity but is a
30+
month of separate work; blocks the migration on a visual detail.
31+
2. **Static HTML injected via template** — fragile; bypasses MyST conventions
32+
and creates a maintenance burden.
33+
3. **`site.parts.footer:` with a `footer.md` file + custom CSS + scienceicons
34+
plugin** — uses MyST's built-in parts mechanism; no custom theme required.
35+
Demonstrated in `tools.scientific-python.org` PR #81 (brianhawthorne,
36+
October 2025). Viable path for the follow-up issue.
37+
4. **Defer with documented issue** — unblocks the migration for SciPy 2026;
38+
footer work is tracked and discoverable.
39+
40+
## Consequences
41+
42+
- The deployed site will have a minimal footer until the follow-up is resolved
43+
- `config.yaml`'s `params.footer` and `params.quicklinks` sections are not
44+
ported to `myst.yml`
45+
- The follow-up issue is linked from `myst.yml` and from the PR description
46+
- Option 3 above (`site.parts.footer:` + CSS) is the recommended implementation
47+
path for the follow-up; `tools.scientific-python.org` PR #81 is the reference
48+
- If the SP community decides MyST theming is too limited, this ADR is a
49+
natural decision point to reconsider the tool choice (see ADR 0001)

‎docs/decisions/0001-myst-migration/0006-sibling-repo-migration.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# ADR 0006 — Sibling SP repos migrate independently
2+
3+
Date: 2026-05-11
4+
Status: Proposed
5+
Branch: lb/myst-migration
6+
7+
## Context
8+
9+
The Scientific Python ecosystem has four Hugo-based sites sharing the same
10+
theme submodule:
11+
12+
| Repo | Domain |
13+
| ----------------------------------------------- | ------------------------------------------- |
14+
| `scientific-python/learn.scientific-python.org` | learn.scientific-python.org ← **this repo** |
15+
| `scientific-python/scientific-python.org` | scientific-python.org |
16+
| `scientific-python/blog.scientific-python.org` | blog.scientific-python.org |
17+
| `scientific-python/tools.scientific-python.org` | tools.scientific-python.org |
18+
19+
Cross-site nav links are plain absolute URLs (not build-time references).
20+
There is no shared build pipeline coupling the repos.
21+
22+
`tools.scientific-python.org` already has an open MyST migration PR
23+
([#81](https://github.com/scientific-python/tools.scientific-python.org/pull/81),
24+
brianhawthorne, opened October 2025, stale as of May 2026). It demonstrates
25+
a working shortcode conversion and a footer implementation using
26+
`site.parts.footer:` + custom CSS (see ADR 0005 option 3).
27+
28+
## Decision
29+
30+
`learn` migrates first. File one tracking issue per sibling repo after this
31+
PR merges, each linking to this PR as a worked example. Sibling repos adopt
32+
MyST on their own schedule.
33+
34+
## Options considered
35+
36+
1. **Migrate all four in lock-step** — synchronizes visual consistency; blocks
37+
`learn` on the slowest-moving repo.
38+
2. **`learn` first; siblings when ready** — proves the pattern; doesn't block
39+
SciPy 2026 deadline.
40+
3. **Wait for MyST theme parity** — defers everything until ADR 0005 follow-up
41+
is resolved; not necessary since content parity is achievable now.
42+
43+
## Consequences
44+
45+
- Cross-site nav continues to work: all links are absolute URLs
46+
- Sibling repos remain on Hugo until they choose to migrate; no visual
47+
breakage to end users
48+
- This PR and `tools.scientific-python.org` PR #81 together form the reference
49+
corpus for sibling repos evaluating MyST migration
50+
- Tracking issues are filed in Phase 8 of the migration plan
51+
- The `scientific-python-hugo-theme` submodule removal in ADR 0003 affects
52+
only this repo; the other three repos remove it in their own migration PRs

‎docs/decisions/0001-myst-migration/0007-deploy-strategy.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# ADR 0007 — Update netlify.toml for MyST build
2+
3+
Date: 2026-05-11
4+
Status: Proposed
5+
Branch: lb/myst-migration
6+
7+
## Context
8+
9+
Both `scientific-python.org` and `learn.scientific-python.org` deploy via
10+
Netlify, which auto-deploys on push to `main` and generates PR preview deploys.
11+
The build command is defined in `netlify.toml`. The current command builds with
12+
Hugo + Dart Sass.
13+
14+
GitHub Actions runs only a lint workflow (`lint.yml`). There is no gh-pages
15+
deploy workflow.
16+
17+
## Decision
18+
19+
Update `netlify.toml` in Phase 6 of the migration: remove the Dart Sass and
20+
Hugo toolchain setup; add `pip install mystmd` before the existing
21+
`make html-all` call. Build command, publish directory, and
22+
`netlify-plugin-checklinks` are otherwise unchanged.
23+
24+
## Options considered
25+
26+
1. **Update `netlify.toml`** — minimal change; keeps Netlify as the deploy
27+
target, PR previews continue to work automatically.
28+
2. **Replace Netlify with gh-pages** — larger change; requires creating new
29+
GitHub Actions deploy workflows, reconfiguring DNS, and losing Netlify PR
30+
previews. Out of scope for this PR.
31+
3. **Keep `netlify.toml` for the checklinks plugin only** — the Netlify
32+
checklinks plugin can be replaced by a `lychee`-based GitHub Actions job
33+
as a separate improvement; not required for this migration.
34+
4. **Drop Netlify entirely; use CircleCI for builds + circleci-artifacts-
35+
redirector-action for PR previews** — demonstrated by
36+
`tools.scientific-python.org` PR #81. Viable but introduces CircleCI
37+
account dependency and is a larger infrastructure change than needed here.
38+
5. **Migrate to Read the Docs** — RTD has first-class MyST/Sphinx support,
39+
built-in PR preview deploys (including for forks), and is already used
40+
widely across the Scientific Python ecosystem. Would replace Netlify
41+
entirely; requires a `.readthedocs.yaml` config and DNS reconfiguration.
42+
Resolves the fork-contributor preview gap (ADR 0007 future work item 1)
43+
as a side effect. Not pursued in this PR — custom domain setup and RTD
44+
account provisioning are out of scope for the migration itself.
45+
46+
## Consequences
47+
48+
- Dart Sass and Hugo version pins removed from `netlify.toml`
49+
- `pip install mystmd` added before `make html-all` in the build command
50+
- Build command (`make html-all`), publish dir (`public/`), and
51+
`netlify-plugin-checklinks` are unchanged
52+
- Netlify auto-deploy and PR previews continue unchanged
53+
54+
## Future work (out of scope)
55+
56+
Two follow-up improvements; both filed as issues before Phase 4 commit 2
57+
(see PLAN.md Phase 4 prerequisite):
58+
59+
- **gh-pages PR preview**: allows contributors working from a fork to
60+
preview builds on their own GitHub Pages without requiring Netlify access.
61+
- **Replace `netlify-plugin-checklinks` with a `lychee`-based GitHub Actions
62+
job**: keeps link checking in CI, removes the Netlify plugin dependency.

0 commit comments

Comments
 (0)