Skip to content

docs(networking): document Console access over AWS PrivateLink#594

Open
david-yu wants to merge 5 commits into
mainfrom
david-yu/console-privatelink-docs
Open

docs(networking): document Console access over AWS PrivateLink#594
david-yu wants to merge 5 commits into
mainfrom
david-yu/console-privatelink-docs

Conversation

@david-yu
Copy link
Copy Markdown
Contributor

@david-yu david-yu commented May 20, 2026

Summary

Adds documentation for accessing Redpanda Console through an AWS PrivateLink VPC endpoint. Today the PrivateLink pages only document Kafka API, Schema Registry, and HTTP Proxy access — Console is silently included when PrivateLink is enabled but isn't mentioned anywhere, so customers either don't realize they can use it or assume they need to hand-roll a Route 53 private hosted zone to make it work.

Engineering confirmed: when PrivateLink is enabled, all endpoints (including Console) are reachable through the VPC endpoint, and DNS is auto-configured via the endpoint service's verified private DNS name — no PHZ override required. This was then verified end-to-end against a private BYOC cluster (see comment below for evidence).

Preview link: https://deploy-preview-594--rp-cloud.netlify.app/redpanda-cloud/networking/configure-privatelink-in-cloud-ui/#access-redpanda-console-from-a-workstation

Changes

  • modules/networking/partials/private-links-access-rp-services-through-vpc.adoc — adds Redpanda Console to the services table and a new === Access Redpanda Console subsection with step-by-step dig + curl verification (used by both the Cloud UI page and the API page).
  • modules/networking/pages/configure-privatelink-in-cloud-ui.adoc — minor wording tweak so the intro of the access section mentions Redpanda Console alongside the other services.

Test plan

  • Netlify preview renders both pages cleanly.
  • console-<id>.<cluster_domain> is reachable on 443 from a consumer VPC with a PrivateLink endpoint to a private BYOC cluster — no PHZ created on the consumer side. (Verified — see comment.)
  • Confirm with engineering that Console is exposed by default when PrivateLink is enabled via the Cloud UI (no extra connect_console toggle required for that path).
  • Confirm whether the How to Connect panel in the Cloud UI currently surfaces the Console URL today; if not, soften the intro sentence accordingly.

🤖 Generated with Claude Code

Adds a Console row to the PrivateLink services table and a new "Access
Redpanda Console" subsection in the shared partial. Engineering confirmed
that DNS for the Console hostname is auto-resolved by the verified
private DNS name on the endpoint service, so consumers do not need to
create a private hosted zone or override DNS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@david-yu david-yu requested a review from a team as a code owner May 20, 2026 03:54
@netlify
Copy link
Copy Markdown

netlify Bot commented May 20, 2026

Deploy Preview for rp-cloud ready!

Name Link
🔨 Latest commit 51c71be
🔍 Latest deploy log https://app.netlify.com/projects/rp-cloud/deploys/6a0de8ea347a59000889d8e1
😎 Deploy Preview https://deploy-preview-594--rp-cloud.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 20, 2026

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8c173548-d62f-415e-9b21-d343c3d8dbc0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR updates PrivateLink VPC endpoint documentation to explicitly include Redpanda Console as an accessible service. Changes clarify that the Cloud Console UI's "How to Connect" section provides URLs for Kafka API, Schema Registry, HTTP Proxy, and Console after PrivateLink is enabled. A shared documentation partial is updated to list Console in the service introduction, expand the ports table to include Console port 443, and add a new "Access Redpanda Console" section with HTTPS URL patterns, DNS resolution behavior notes, and VPC endpoint security group configuration requirements.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Possibly related PRs

  • redpanda-data/cloud-docs#431: Earlier PrivateLink documentation expansion that references BYOC private/public API gateway access for Redpanda Console endpoints, directly overlapping with this PR's Console URL and port documentation.
  • redpanda-data/cloud-docs#504: Prior PrivateLink Cloud UI documentation update to the same shared partial and page, laying groundwork for expanded Console service details now being documented.

Suggested reviewers

  • micheleRP
  • paulzhang97
  • simonlord
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding documentation for Console access over AWS PrivateLink, which directly corresponds to the primary objective of the pull request.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The PR description provides comprehensive information covering the purpose (documenting Console access over PrivateLink), background context, specific changes made, test plan with verification evidence, and page previews.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch david-yu/console-privatelink-docs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
modules/networking/partials/private-links-access-rp-services-through-vpc.adoc (1)

70-70: ⚡ Quick win

Tighten the security group guidance to avoid over-permissive rules.

At Line 70, consider explicitly stating inbound TCP 443 should be allowed from client workload sources only (for example, consumer VPC CIDRs or specific client security groups), not broad sources.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@modules/networking/partials/private-links-access-rp-services-through-vpc.adoc`
at line 70, Update the guidance line that currently says "The security group
attached to your VPC endpoint must allow inbound TCP on port `443`" to specify
that the allowance must be restricted to client workload sources only (e.g.,
consumer VPC CIDRs or explicit client security groups) rather than broad sources
like 0.0.0.0/0; mention examples (consumer VPC CIDRs, specific client security
group IDs) and recommend avoiding overly-permissive source ranges to tighten
access control for the VPC endpoint security group.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@modules/networking/partials/private-links-access-rp-services-through-vpc.adoc`:
- Line 70: Update the guidance line that currently says "The security group
attached to your VPC endpoint must allow inbound TCP on port `443`" to specify
that the allowance must be restricted to client workload sources only (e.g.,
consumer VPC CIDRs or explicit client security groups) rather than broad sources
like 0.0.0.0/0; mention examples (consumer VPC CIDRs, specific client security
group IDs) and recommend avoiding overly-permissive source ranges to tighten
access control for the VPC endpoint security group.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1787260d-2239-4371-a2d1-3d71fee4bbe7

📥 Commits

Reviewing files that changed from the base of the PR and between 9e7c687 and d60e868.

📒 Files selected for processing (2)
  • modules/networking/pages/configure-privatelink-in-cloud-ui.adoc
  • modules/networking/partials/private-links-access-rp-services-through-vpc.adoc

Verified end-to-end against a private BYOC cluster with a PrivateLink
endpoint in a consumer VPC:

- Cluster DNS publicly returns service-side LB IPs (10.0.x.x).
- From inside the consumer VPC, dig returns the endpoint's ENI private
  IP (e.g. 172.31.0.97) via the verified private DNS name
  `*.<cluster_domain>` registered on the endpoint service.
- curl returns HTTP 200 with title "Redpanda Console" served over
  HTTPS through the endpoint.

Corrects the Console URL form from `console.<cluster_domain>` to
`console-<id>.<cluster_domain>` (the random per-cluster id suffix the
control plane assigns), and points to the *How to Connect* section as
the authoritative source.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@david-yu
Copy link
Copy Markdown
Contributor Author

Verification: Console over PrivateLink works end-to-end

Tested against a private BYOC cluster (d86eirghvfrjvm53j6vg, AWS us-east-2) with connection_type = "private" and aws_private_link.connect_console = true. Consumer-side VPC endpoint created in the same AWS account using the default VPC (172.31.0.0/16); no Route 53 private hosted zone, no DNS overrides.

Public DNS (control: from outside the VPC)

dig on a laptop returns the service-side LB IPs in Redpanda's VPC. curl times out because those private IPs aren't routable from the public internet — i.e. the cluster is genuinely private.

$ dig +short console-65e0163a.d86eirghvfrjvm53j6vg.byoc.prd.cloud.redpanda.com
10.0.10.222
10.0.8.70
10.0.6.43

$ curl -I --max-time 8 https://console-65e0163a.d86eirghvfrjvm53j6vg.byoc.prd.cloud.redpanda.com
curl: (28) Connection timed out after 8006 milliseconds

Verified private DNS on the endpoint service

The PrivateLink endpoint service publishes a wildcard verified private DNS name that covers the Console hostname:

$ aws ec2 describe-vpc-endpoint-services --query "ServiceDetails[?ServiceName=='com.amazonaws.vpce.us-east-2.vpce-svc-0c4382ee450e3c281'].PrivateDnsName"
*.d86eirghvfrjvm53j6vg.byoc.prd.cloud.redpanda.com

This is what makes auto-resolution work in consumer VPCs without any PHZ.

From inside the consumer VPC

With private_dns_enabled = true on the VPC endpoint, DNS resolves the Console hostname to the consumer-side ENI private IP (different subnet from the service-side IPs above), and HTTPS GET returns 200 from Redpanda Console:

$ dig +short console-65e0163a.d86eirghvfrjvm53j6vg.byoc.prd.cloud.redpanda.com
172.31.0.97

$ curl -sS -o /tmp/console.html -w "HTTP %{http_code}  ip=%{remote_ip}\n" \
    https://console-65e0163a.d86eirghvfrjvm53j6vg.byoc.prd.cloud.redpanda.com/
HTTP 200  ip=172.31.0.97

$ grep -oE "<title>[^<]+</title>" /tmp/console.html
<title>Redpanda Console</title>

The seed Kafka API hostname also resolves to the same ENI (172.31.0.97), confirming the wildcard private DNS name covers every Redpanda service on the endpoint.

Still open

  • Whether the How to Connect panel in the Cloud UI actually displays the Console URL today. If not, the intro sentence on configure-privatelink-in-cloud-ui.adoc should be softened to "Redpanda Console is reachable at https://console-<id>.<cluster_domain>" rather than implying it appears in the panel.
  • Whether the connect_console field is always on by default when PrivateLink is enabled via the Cloud UI (today's repro had it on via Terraform).

Adds a new section to the Cloud UI page describing how to access
Redpanda Console for a private BYOC cluster from a workstation outside
the client VPC. Verified against a real cluster.

Key points captured:
- Cluster Console for a private BYOC cluster is not browsable directly;
  the SPA loads but has no standalone login form. Authentication is
  handed off from Redpanda Cloud Console.
- The cluster Overview page on cloud.redpanda.com is served by the
  control plane and works without a VPN.
- The cluster's left navigation (Topics, Brokers, Consumer groups) is
  served by Redpanda Console behind PrivateLink, so it only loads when
  the workstation has a VPN session into the client VPC.
- AWS Client VPN settings specific to PrivateLink: client CIDR must not
  overlap VPC CIDR, DNS server IPs must point at the VPC's resolver
  (so Console hostnames resolve via the endpoint ENI), split-tunnel
  enabled.

Softens the partial's Console section from "browse to the URL" to a
network-path verification, with a forward reference to the workstation
flow on the Cloud UI page.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@david-yu
Copy link
Copy Markdown
Contributor Author

Verified workflow: workstation access via AWS Client VPN

Tested end-to-end against the same private BYOC cluster used for the earlier verification.

Standing up Client VPN

Used terraform to stand up an AWS Client VPN endpoint in the consumer VPC: mutual TLS auth (self-signed CA + server + client cert via the tls provider, server cert + CA uploaded to ACM), split-tunnel enabled, dns_servers = [172.31.0.2] (the default VPC's resolver), authorization rule allowing the client CIDR (10.100.0.0/22) to reach the VPC CIDR. Endpoint took ~5 minutes to reach available after subnet association. Generated .ovpn was loaded into the official AWS VPN Client (macOS).

Confirming DNS pushes through correctly

From the Mac with the VPN connected:

$ dig +short console-65e0163a.d86eirghvfrjvm53j6vg.byoc.prd.cloud.redpanda.com
172.31.0.97

This is the PrivateLink endpoint's ENI IP, not the cluster VPC's internal LB. Confirms the Client VPN endpoint pushed 172.31.0.2 (default VPC resolver) as the DNS server, and the resolver returned the verified-private-DNS answer via the endpoint.

Browser behavior

  • Going directly to https://console-65e0163a.<...> shows "you need to log in" with no login form. As expected — Redpanda Console for a private BYOC cluster does not expose a standalone login form; the auth token is handed off from Redpanda Cloud Console.
  • Going to https://cloud.redpanda.com → cluster → Overview works without the VPN (Cloud control plane).
  • With the VPN connected, the cluster's left navigation (Topics, Brokers, Consumer groups, etc.) loads. Those views are served by Redpanda Console at https://console-<id>.<cluster_domain>, and the browser can now resolve and reach that hostname over PrivateLink.
  • Confirmed end-to-end: created a topic from the Cloud Console UI with the VPN connected.

There is no "Open Console" button — the cluster's left navigation IS the Console UI, embedded in the Cloud Console cluster page.

Updates in cfe5384

  • Adds == Access Redpanda Console from a workstation on the Cloud UI page with three subsections: Set up AWS Client VPN, Connect using AWS VPN Client, Open Redpanda Console through the Cloud Console left navigation.
  • Calls out the four Client-VPN-specific settings that matter for PrivateLink: client CIDR, DNS server IPs, split-tunnel, target network association.
  • Softens the partial's === Access Redpanda Console section from "browse to the URL" to "verify the network path" with a forward reference to the workstation flow. The dig/curl verification block is retained as a sanity check.

@david-yu david-yu requested a review from paulzhang97 May 20, 2026 06:45
ASCII diagram in the workstation access section showing the two-hop
flow: Client VPN puts the laptop into the client VPC, then PrivateLink
carries the connection to the cluster. Clarifies that Client VPN itself
does not use PrivateLink — the VPN is just one of several ways a client
can reach the VPC, and PrivateLink is the constant once you're inside.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread modules/networking/pages/configure-privatelink-in-cloud-ui.adoc Outdated
Comment thread modules/networking/pages/configure-privatelink-in-cloud-ui.adoc Outdated
- Drop "only" in the workstation intro; Console is reachable through
  other private routes too (VPC peering, transit gateway). Per
  @paulzhang97 review.
- Widen the Client VPN inbound port range from 30000-30999 to
  30000-35999 to cover both seed and per-broker Kafka API and HTTP
  Proxy ports. Per @paulzhang97 review.
- Tighten partial's port 443 SG guidance to scope inbound to client
  workload sources (consumer VPC CIDR or specific client security
  groups) and call out that broad source ranges like 0.0.0.0/0 should
  be avoided. Per @coderabbitai nitpick.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@david-yu
Copy link
Copy Markdown
Contributor Author

Thanks @paulzhang97 resolved the outstanding issues and sent over for re-review

@david-yu david-yu requested a review from paulzhang97 May 20, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants