Skip to content

feat(website): add docs search#6496

Open
emmahodcroft wants to merge 9 commits into
mainfrom
add-search-deps
Open

feat(website): add docs search#6496
emmahodcroft wants to merge 9 commits into
mainfrom
add-search-deps

Conversation

@emmahodcroft
Copy link
Copy Markdown
Member

@emmahodcroft emmahodcroft commented May 22, 2026

Why

It's quite hard to find things in our docs — the menu on the left is quite overwhelming to read through. If you know exactly what you want, it's ok, but for a newcomer I imagine it's quite intimidating.

@emmahodcroft, pathoplexus/pathoplexus#974

So let's fix it. This adds a client-side search bar at the top of the /docs and /about menus.

What's here

A MiniSearch-powered search across all MDX content in src/pages/docs/** and src/pages/about/**:

  • Fuzzy + prefix matching — finds "revison" → "revision", "subm" → "submission".
  • Results grouped by section (Documentation / About).
  • Built at build time, not per request. buildSearchIndex.ts uses import.meta.glob('?raw') so Vite bundles the MDX content into the server output; the {options, documents} JSON payload is JSON.stringify-ed once at module load. The /search-index.json route is a one-liner that hands the precomputed string back — no per-request work, no Cache-Control / ETag plumbing needed.
  • Works for downstream consumers (e.g. Pathoplexus) automatically. The glob resolves at their build time against the merged filesystem (sync overlay + loculus tree), so any extra MDX pages they add get indexed without further changes.

Frontmatter is parsed with the yaml library (tolerant of CRLF), so titles render correctly regardless of which OS authored the file.

UI niceties picked up along the way:

  • Input is wrapped in DisabledUntilHydrated so Playwright / early keystrokes can't race hydration (per website/AGENTS.md).
  • "Loading…" indicator lives inside the absolutely-positioned dropdown rather than below the input → no layout shift on mount.
  • Custom focus state on the wrapper (focus-within:border-primary-400) so the input doesn't show the browser's default blue ring.

Files

File What it does
website/src/utils/buildSearchIndex.ts Reads MDX via import.meta.glob('?raw'), parses frontmatter with yaml, produces the JSON payload.
website/src/pages/search-index.json.ts One-line GET route returning the precomputed string.
website/src/components/Navigation/DocsSearch.tsx Search bar + dropdown + result grouping.
website/src/layouts/{Doc,About}Layout.astro Render <DocsSearch client:load /> above the menu.
website/package.json / package-lock.json Adds minisearch + yaml.

🚀 Preview: Add preview label to enable

@claude claude Bot added website Tasks related to the web application dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code labels May 22, 2026
emmahodcroft and others added 2 commits May 22, 2026 16:01
Adds the missing minisearch entry to package-lock.json so `npm ci`
no longer fails. yaml was already present as a transitive dep so
this is mostly a no-op for yaml (just adds it to the root deps list).
Adds a MiniSearch-powered client-side search that's exposed as a bar
at the top of the docs and about menu.

- buildSearchIndex.ts pulls MDX content at build time via
  import.meta.glob('?raw') so Vite bundles the source into the
  server output. The {options, documents} payload is JSON.stringify-ed
  once when the Node module loads; the HTTP route just hands the
  precomputed string back, so there's no per-request work and no
  need for Cache-Control / ETag headers.
- Frontmatter titles are parsed with the yaml lib.
- DocLayout.astro and AboutLayout.astro render <DocsSearch client:load />
  above the existing DocsMenu.

This lands a feature that was originally drafted in pathoplexus PR
pathoplexus/pathoplexus#974; moving the impl to Loculus so downstream
consumers automatically get the same behaviour without overlay files.
Comment thread website/src/utils/buildSearchIndex.ts Fixed
theosanderson-agent pushed a commit to pathoplexus/pathoplexus that referenced this pull request May 22, 2026
The DocsSearch component, buildSearchIndex util and search-index.json
route all moved upstream in loculus-project/loculus#6496, which also
wires the search bar into DocLayout and AboutLayout. Drop the
overlaid copies here and bump loculusVersion to the new tip
(b773fe7ab) so the sync pulls everything in.
- buildSearchIndex frontmatter regex tolerated only LF line endings,
  so mdx files saved with CRLF (e.g. pathoplexus'
  about/governance/data-submission.mdx) failed to parse and showed up
  as "Untitled".
- DocsSearch was rendering a "Loading search..." line below the input
  while the index was loading, which shifted the page content as the
  text appeared and disappeared. Move that indicator into the
  absolutely-positioned results dropdown so it only renders when the
  user is actually searching (no CLS).
@theosanderson theosanderson changed the title Add minisearch & yaml to deps feat(website): add docs search May 22, 2026
claude and others added 2 commits May 22, 2026 16:31
The bare input rendered the browser's default blue focus outline. Add
focus:outline-none focus:ring-0 to suppress it, and give the
surrounding wrapper a focus-within:border-primary-400 so the search
bar still has a clear themed focus indicator.
…racter sanitization'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@theosanderson
Copy link
Copy Markdown
Member

@codex review

Comment thread website/src/utils/buildSearchIndex.ts Fixed
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fa7a73250a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread website/src/components/Navigation/DocsSearch.tsx Outdated
claude added 3 commits May 22, 2026 16:38
The content field is only fed into MiniSearch for text matching and
is never rendered as HTML (search results show titles only), so the
multi-pass HTML-comment sanitization Copilot autofixed wasn't
buying any safety and CodeQL still flagged it. Removing the rule
entirely keeps the function single-pass and resolves the alert.
website/AGENTS.md asks interactive elements to be wrapped with
DisabledUntilHydrated (or gated on useClientFlag) so Playwright /
hydration races can't drop early keystrokes. Wrap the search input
accordingly. Codex review flagged this.
The index is computed once at module load, so it's essentially a
static asset for the lifetime of the running server. Compute a sha1
ETag alongside the JSON, send Cache-Control: public, max-age=3600,
and respond 304 to If-None-Match when the hash hasn't changed.

Means browsers cache the index for an hour, and on revalidation we
short-circuit to a 304 instead of re-sending the payload.
@theosanderson
Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code website Tasks related to the web application

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants