feat: universal proxy (new approach) by cjroth · Pull Request #837 · thunderbird/thunderbolt

cjroth · 2026-05-06T04:14:26Z

Note

High Risk
High risk because it changes security-sensitive proxy/CORS behaviour (SSRF protections, header forwarding, redirect handling) and introduces a new WS proxy path; mistakes could leak credentials or open SSRF vectors.

Overview
Switches the backend to a new universal proxy implementation: /v1/proxy now supports header passthrough via X-Proxy-Passthrough-*, redirect controls, streaming with caps/timeouts, SSRF/DNS pinning, and adds a new /v1/proxy/ws relay with subprotocol-encoded targets.

Adds proxy observability via proxy_request/proxy_ws_relay structured logs and OpenTelemetry span attributes, explicitly redacting full URLs/credentials while recording hostname, bytes, duration, status, and categorical error types.

Introduces new authenticated APIs: /v1/search (Exa-backed, normalises results to HTTPS and derives favicons) and /v1/preview (POST-only OG metadata extraction with HTTPS image upgrade and per-user 10‑minute private caching), plus new e2e tests.

Removes the old /pro/proxy, /pro/link-preview, and /mcp-proxy routes and their tests, updates CORS to allowedHeaders: true (echo requested headers) and refreshes corsExposeHeaders defaults/docs/env examples accordingly; also tweaks auth creation for test DI, sets Better Auth baseURL, and adjusts dev tooling/lint globals/logger exports.

^{Reviewed by Cursor Bugbot for commit c214826. Bugbot is set up for automated code reviews on this repo. Configure here.}

github-actions · 2026-05-06T04:15:07Z

Semgrep Security Scan

No security issues found.

github-actions · 2026-05-06T04:20:42Z

PR Metrics

Metric	Value
Lines changed (prod code)	`+2289 / -1266`
JS bundle size (gzipped)	🟢 1.02 MB → 1.02 MB (-3.5 KB, -0.3%)
Test coverage	🟢 71.47% → 72.09% (+0.6%)
Performance (preview)	Preview not ready — Render deploy may have timed out
Accessibility	—
Best Practices	—
SEO	—

Updated Thu, 14 May 2026 19:35:22 GMT · run #1499

…from ending up in server logs

- inject DnsLookup into url-validation, proxy, and preview routes - inject SearchExaClient into search route via createApp deps - inject email senders into createAuth to capture OTPs per test - avoids mock.module leaking across test files

- store test PGlite instance on globalThis so it survives module reloads - close on process beforeExit instead of afterAll, so reruns reuse one instance rather than reinitializing (and re-running migrations) each time

github-actions · 2026-05-06T19:18:32Z

Preview environment deployed 🚀

Service	URL
Marketing / blog / docs	https://thunderbolt-pr-837.preview.thunderbolt.io
App	https://app-pr-837.preview.thunderbolt.io
API	https://api-pr-837.preview.thunderbolt.io
Keycloak	https://auth-pr-837.preview.thunderbolt.io
PowerSync	https://powersync-pr-837.preview.thunderbolt.io

Stack: preview-pr-837 · Commit: c214826744a06192c9ad5f931e1af6c49d79c2b2

Auto-destroys on PR close/merge. Login via the bundled Keycloak realm — demo@thunderbolt.io / demo by default.

- Eliminates a sign-in race where the bearer is returned before the session row is visible, which surfaced as a confusing 401 on the first authenticated request in tests.

- move cleanup into bun:test afterAll so it runs inside the test runner lifecycle, before Bun terminates worker threads - drop the globalThis-shared PGlite instance; one instance per process is sufficient now that cleanup is reliable

…xpose-headers)

…PostHog

…hrough) - observability: strip PostHog (Pino + OTel only); add ProxyErrorType union and `trace.getActiveSpan().setAttributes(...)` for proxy.* attrs. Wire real status / duration_ms / bytes_in / bytes_out from the response and capStream (fixes bot MEDIUM findings: hardcoded status, zero duration/bytes). Tag error_type on every failure path. - content-encoding passthrough: pass `decompress: false` to Bun fetch, remove `content-encoding` from DROPPED_RESPONSE_HEADERS. Browser now decodes; the proxy holds compressed bytes only. Comment near 10MB cap documents the zip-bomb risk shift to the caller. - streaming: capStream returns { stream, bytesRead() } and fires a single `onComplete(bytes)` so observability emits exactly once after stream drain (graceful, cap-hit, idle, error, or cancel).

For streaming POSTs (no follow-redirects), bytesIn was captured at the moment fetchFn() returned response headers — before the request body had fully drained through capStream. Replace the captured number with a late-read getter so the observability emission (which fires from the response stream's onComplete) sees the final upload byte count.

- routes.ts: replace unbounded arrayBuffer() in buffered-body path with bounded streaming accumulator that early-terminates at maxBodyBytes. Closes a DoS vector where a chunked upload (no Content-Length) would materialise the full body in memory before the 413 check fired. - routes.ts: document Bun's decompress:false semantics (Bun >=1.3 keeps content-encoding on the Response even after auto-decode, so the cast is load-bearing for correctness — not just a perf flag). - routes.test.ts: add streaming-POST bytes_in regression test exercising the validator's deferred-getter fix (commit 7d5ddada) directly. - routes.test.ts: add chunked-upload DoS test proving early-termination.

- Remove backend/scripts/dev.sh (Chris flagged out-of-scope) - Revert backend/src/db/client.ts dev rewrite - Revert playwright.config.ts tweak - Revert powersync localhost defaults in settings.ts (closes HIGH bot finding) These items moved to italomenezes/cjroth-proxy-dev-env (stacked PR).

# Conflicts: # backend/.env.example # backend/eslint.config.js # backend/src/auth/auth.ts # backend/src/matchers.d.ts # backend/src/pro/link-preview.ts # backend/src/proxy/routes.ts # backend/src/utils/streaming.ts # backend/src/utils/url-validation.ts # powersync-service/docker-compose.yml

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 94342ac. Configure here.}

cjroth · 2026-05-14T17:21:33Z

 }

+/**
+ * Memoized `proxyFetch` keyed on `cloudUrl`. Construction is cheap, but creating


I think we should move this into React context instead of just let vars

Also, what is the use case for having multiple cloud URLs at the same time? I feel this could be simplified by having a simple useState inside of the React context that is added in this commit.

cjroth · 2026-05-14T17:25:35Z

  return proxyFetch
 }

+/** Test-only: clears the module-scoped proxy-fetch cache so tests start from a known state. */


This could be removed if we can use useState

cjroth · 2026-05-14T17:54:04Z

+
+import { fetch as tauriFetch } from '@tauri-apps/plugin-http'
+import {
+  PASSTHROUGH_PREFIX,


…xy toggle

…text, preferences

…o module consts

…ticated

cjroth added 4 commits May 5, 2026 00:13

refactor: universal proxy

5bc10f2

fix: multiple dev environment fixes / improvements

45deabb

fix: dev environment config

d0c6a62

fix: vite require-corp blocking images + confusing 401 status from proxy

ff32a8e

github-code-quality Bot found potential problems May 6, 2026

View reviewed changes

Comment thread backend/src/db/client.ts Fixed

fix: noisy errors

7593271

cursor Bot reviewed May 6, 2026

View reviewed changes

Comment thread backend/src/proxy/routes.ts Outdated

Comment thread backend/src/api/search.ts Outdated

Comment thread backend/src/api/search.ts Outdated

cjroth had a problem deploying to preview May 6, 2026 04:18 — with GitHub Actions Failure

cjroth added 2 commits May 6, 2026 00:21

fix: use request body / post for url preview endpoint to prevent url …

ad8f4ca

…from ending up in server logs

fix: e2e tests

6592b6b

cjroth had a problem deploying to preview May 6, 2026 04:26 — with GitHub Actions Failure

fix: lint

6c495a7

cursor Bot reviewed May 6, 2026

View reviewed changes

Comment thread backend/src/proxy/routes.ts Outdated

cjroth had a problem deploying to preview May 6, 2026 04:43 — with GitHub Actions Failure

cjroth added 2 commits May 6, 2026 01:16

feat: e2e test

e7701e7

cursor Bot reviewed May 6, 2026

View reviewed changes

Comment thread backend/src/api/preview.ts

cjroth had a problem deploying to preview May 6, 2026 06:41 — with GitHub Actions Failure

refactor: share PGlite across rerun-each to avoid RSS leak

650e0bc

- store test PGlite instance on globalThis so it survives module reloads - close on process beforeExit instead of afterAll, so reruns reuse one instance rather than reinitializing (and re-running migrations) each time

cursor Bot reviewed May 6, 2026

View reviewed changes

Comment thread backend/src/config/logger.ts Outdated

cjroth had a problem deploying to preview May 6, 2026 18:37 — with GitHub Actions Failure

ci: retrigger workflows

b382a37

cjroth temporarily deployed to preview May 6, 2026 19:15 — with GitHub Actions Inactive

cursor Bot reviewed May 6, 2026

View reviewed changes

Comment thread backend/src/config/settings.ts Outdated

test: verify session row exists before returning e2e bearer

bda1429

- Eliminates a sign-in race where the bearer is returned before the session row is visible, which surfaced as a confusing 401 on the first authenticated request in tests.

cjroth temporarily deployed to preview May 6, 2026 19:26 — with GitHub Actions Inactive

ital0 added 8 commits May 12, 2026 19:41

bucket-C: restore CORS regressions (PostHog allowHeaders + protocol e…

ebf3fb0

…xpose-headers)

bucket-C: refresh CORS docs/comments to match allowedHeaders:true on …

4acfb60

…PostHog

fix(bucket-B2): rename test helper to satisfy naming-convention lint

8013d2d

bucket-E: proxy_enabled toggle

4584966

style: reformat proxy files to match prettier line-width

3507897

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread backend/src/api/search.ts

Comment thread backend/src/config/settings.ts

ital0 added 2 commits May 13, 2026 15:32

refactor: rename SCREAMING_SNAKE proxy-protocol constants to camelCase

7c862d2

refactor(ws): add error_type to WS observability paths

4c6aeb6

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread backend/src/index.ts

ital0 added 2 commits May 13, 2026 15:51

cursor Bot reviewed May 13, 2026

View reviewed changes

Comment thread backend/src/config/settings.ts

ital0 temporarily deployed to preview May 13, 2026 20:03 — with GitHub Actions Inactive

ital0 added 2 commits May 13, 2026 17:47

fix(proxy): drain response bodies in tests to prevent timer leak

e46a256

fix(e2e): bypass backend dev.sh (moved to stacked PR)

a9c2d2b

ital0 temporarily deployed to preview May 13, 2026 20:55 — with GitHub Actions Inactive

ital0 temporarily deployed to preview May 14, 2026 15:34 — with GitHub Actions Inactive

ital0 force-pushed the cjroth/universal-proxy branch from 531b41c to a9c2d2b Compare May 14, 2026 15:43

ital0 temporarily deployed to preview May 14, 2026 15:46 — with GitHub Actions Inactive

cjroth commented May 14, 2026

View reviewed changes

ital0 added 4 commits May 14, 2026 16:28

chore(preferences): drop misleading "When disabled" copy on cloud pro…

80096ac

…xy toggle

refactor(proxy): share computeEffectiveProxyEnabled across fetch, con…

4b9aea9

…text, preferences

refactor(proxy): hoist default args of computeEffectiveProxyEnabled t…

cfdbe7e

…o module consts

feat(preferences): disable cloud proxy toggle when user is not authen…

c214826

…ticated

ital0 temporarily deployed to preview May 14, 2026 19:33 — with GitHub Actions Inactive

Conversation

cjroth commented May 6, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 6, 2026

Semgrep Security Scan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Metrics

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cjroth May 14, 2026

Choose a reason for hiding this comment

Uh oh!

cjroth May 14, 2026

Choose a reason for hiding this comment

Uh oh!

cjroth May 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cjroth May 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cjroth commented May 6, 2026 •

edited by cursor Bot

Loading

github-actions Bot commented May 6, 2026 •

edited

Loading

github-actions Bot commented May 6, 2026 •

edited

Loading