From a2ee5ebeab2e6c6cbbbf846203dff47919d75e84 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 1 May 2026 17:21:12 -0700 Subject: [PATCH 01/14] Add quickstart draft --- .../sql/pages/get-started/sql-quickstart.adoc | 167 ++++++++++++++++++ 1 file changed, 167 insertions(+) create mode 100644 modules/sql/pages/get-started/sql-quickstart.adoc diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc new file mode 100644 index 000000000..20c706da4 --- /dev/null +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -0,0 +1,167 @@ += Redpanda SQL quickstart +:description: Connect to Redpanda SQL on a BYOC cluster and run your first query on streaming data. +:page-topic-type: guide + +Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda BYOC. It lets you query streaming data in your Redpanda topics with standard SQL, without building ETL pipelines or deploying a separate analytics system. In this quickstart, you connect with `psql` and run your first query against a Redpanda topic. + +== Prerequisites + +* A Redpanda BYOC cluster on AWS with Redpanda SQL enabled. See xref:sql:get-started/deploy-sql-cluster.adoc[]. +* A Redpanda topic with a schema registered in Schema Registry. If you don't have one, follow the optional <> section below to create a sample `orders` topic. +* https://www.postgresql.org/download/[`psql`^] (PostgreSQL client) installed on your local machine. + +// TODO: Verify the exact connection string format and where users get credentials. +// From PRD: SCRAM auth preserved, connection string available in Cloud Console and API response. +// Confirm with engineering what SCRAM credentials does the user use - superuser auto-created by Control Plane? + +== Get connection details + +After Redpanda SQL is provisioned on your cluster: + +. In the Redpanda Cloud Console, go to your cluster and open the *SQL* tab. +. Copy the connection string. + +== Connect to Redpanda SQL + +Use `psql` to connect to the SQL engine. Paste the connection string you copied from the Console: + +// TODO: Replace with actual connection string format once confirmed with engineering. +[,bash] +---- +psql "" +---- + +On a successful connection, you should see output similar to: + +// TODO: Verify current psql banner text. +[.no-copy] +---- +psql (14.x, server 16.0) +SSL connection (protocol: TLSv1.3) +Type "help" for help. + +=> +---- + +[#optional-produce-sample-data] +== (Optional) Produce sample data + +[TIP] +==== +Skip this section if you already have a Redpanda topic with a schema registered in Schema Registry that you want to query. +==== + +If you don't have a schema-registered topic to query yet, follow these steps to create an `orders` topic with a small set of sample records. Redpanda SQL reads the topic's schema from Schema Registry to map fields to SQL columns, so the topic must have a registered schema before you can query it. + +This section uses xref:reference:rpk/index.adoc[`rpk`], which you can install by following xref:get-started:rpk-install.adoc[]. You also need a user with permissions to create topics, register schemas, and produce records. + +. Create a topic: ++ +[,bash] +---- +rpk topic create orders +---- + +. Save the following Protobuf schema as `order.proto`: ++ +// TODO: Confirm the GA-supported schema format(s) with engineering. If JSON Schema is supported at GA, consider switching this example to JSON for simpler UX. +[,proto] +---- +syntax = "proto3"; + +message Order { + int64 order_id = 1; + string customer = 2; + string product = 3; + int64 amount = 4; // amount in cents + string status = 5; // "pending", "shipped", "completed" +} +---- + +. Register the schema against the topic's value subject: ++ +[,bash] +---- +rpk registry schema create orders-value --schema order.proto +---- + +. Produce a few sample records. The `--schema-id=topic` flag tells `rpk` to use the topic name strategy to look up the schema you just registered: ++ +[,bash] +---- +rpk topic produce orders --schema-id=topic < 10000 +ORDER BY amount DESC +LIMIT 20; +---- + +== Inspect your SQL cluster + +Redpanda SQL provides built-in commands to inspect the state of your SQL cluster: + +[,sql] +---- +SHOW NODES; -- List SQL compute nodes and their status +SHOW QUERIES; -- List currently running queries +---- + +== Next steps + +* xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. +* xref:sql:connect-to-sql/language-clients/psycopg2.adoc[Connect with Python (psycopg2)]: Query Redpanda SQL programmatically. +* xref:sql:connect-to-sql/language-clients/java-jdbc.adoc[Connect with Java (JDBC)]: Integrate with Java applications. From 9fe925a27fa719aa91e128a48428899c59ed7f7d Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 1 May 2026 17:21:48 -0700 Subject: [PATCH 02/14] Add deploy doc (use case 1) --- .../pages/get-started/deploy-sql-cluster.adoc | 162 ++++++++++++++++++ 1 file changed, 162 insertions(+) create mode 100644 modules/sql/pages/get-started/deploy-sql-cluster.adoc diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc new file mode 100644 index 000000000..76f5494ba --- /dev/null +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -0,0 +1,162 @@ += Enable Redpanda SQL on a BYOC cluster +:description: Enable the Redpanda SQL engine on a BYOC cluster so that users can query streaming data with standard PostgreSQL syntax. +:page-topic-type: how-to + +Enable Redpanda SQL on a BYOC cluster to give your team the ability to query streaming data in Redpanda topics using standard PostgreSQL syntax. + +== Prerequisites + +To enable Redpanda SQL, you need: + +* Admin permissions in your Redpanda Cloud organization. +* For the Cloud API path, a valid bearer token for the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. See link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. + +== Enable Redpanda SQL + +You can enable Redpanda SQL when you create a new BYOC cluster or on an existing cluster. + +=== On a new cluster + +[tabs] +===== +Cloud Console:: ++ +-- +. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. +. Start creating a new BYOC cluster on AWS. For details and prerequisites, see xref:get-started:cluster-types/byoc/aws/create-byoc-cluster-aws.adoc[]. +. In the cluster creation form, select the option to enable SQL. +. Complete the remaining cluster configuration and deploy. +-- + +Cloud API:: ++ +-- +. Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. +. Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[`POST /v1/clusters`] request with `oxla.enabled` set to `true` in the cluster spec: ++ +[,bash] +---- +curl -X POST "https://api.redpanda.com/v1/clusters" \ + -H "Authorization: Bearer $AUTH_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "cluster": { + "name": "", + "cloud_provider": "CLOUD_PROVIDER_AWS", + "type": "TYPE_BYOC", + "region": "", + "zones": [ ], + "throughput_tier": "", + "resource_group_id": "", + "oxla": { + "enabled": true + } + } + }' +---- ++ +For the full request body and field reference, see the link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[Create cluster API]. +. The request returns the ID of a long-running operation. Poll the link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] endpoint until the operation completes. +-- +===== + +=== On an existing cluster + +To enable, scale, or disable SQL on an existing cluster, you also need the cluster ID, which you can find in the *Details* section of the cluster overview in the Cloud Console. + +. Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. +. Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request, replacing `{cluster.id}` with your cluster ID: ++ +[,bash] +---- +curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ + -H "Authorization: Bearer $AUTH_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"oxla":{"enabled":true}}' +---- ++ +The request returns the ID of a long-running operation. Poll the link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] endpoint until the operation completes: ++ +[,bash] +---- +curl -X GET "https://api.redpanda.com/v1/operations/{operation.id}" \ + -H "Authorization: Bearer $AUTH_TOKEN" \ + -H "Content-Type: application/json" +---- ++ +When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. + +== Scale Redpanda SQL + +Redpanda SQL supports horizontal scaling from 1 to 12 nodes per cluster. Scaling to 0 is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. + +[tabs] +===== +Cloud Console:: ++ +-- +. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. +. Go to your BYOC cluster and open the *SQL* tab. +. Set the node count to a value between 1 and 12, then save. +-- + +Cloud API:: ++ +-- +Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with the new replica count. Replace `{cluster.id}` with your cluster ID and `` with a value between 1 and 12: + +[,bash] +---- +curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ + -H "Authorization: Bearer $AUTH_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"oxla":{"replicas":}}' +---- + +The request returns the ID of a long-running operation. Poll link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] until the operation completes. +-- +===== + +== Verify the SQL engine is running + +After you enable Redpanda SQL, provisioning may take several minutes. To verify the SQL engine is running: + +// TODO: Confirm with engineering if there are any other specific status indicators that users can see in the Console when SQL is running. Is the SQL tab only visible when SQL is enabled? + +== Disable Redpanda SQL + +[WARNING] +==== +Disabling Redpanda SQL purges the stored catalog state for the SQL engine and deletes its data from object storage. In-flight queries fail when SQL is disabled. To temporarily stop SQL compute without losing state, <> instead. +==== + +[tabs] +===== +Cloud Console:: ++ +-- +. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. +. Go to your BYOC cluster and open the *SQL* tab. +. Click *Remove* and confirm. +-- + +Cloud API:: ++ +-- +Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with `oxla.enabled` set to `false`. Replace `{cluster.id}` with your cluster ID: + +[,bash] +---- +curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ + -H "Authorization: Bearer $AUTH_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"oxla":{"enabled":false}}' +---- + +The request returns the ID of a long-running operation. Poll link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] until the operation completes. +-- +===== + +== Next steps + +* xref:sql:get-started/sql-quickstart.adoc[Quickstart]: Connect to Redpanda SQL with `psql` and run your first query. From b787bf7ca11a565dd68e02cd629d024974c7cea8 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Fri, 1 May 2026 17:21:56 -0700 Subject: [PATCH 03/14] Add to nav --- modules/sql/pages/get-started/index.adoc | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 modules/sql/pages/get-started/index.adoc diff --git a/modules/sql/pages/get-started/index.adoc b/modules/sql/pages/get-started/index.adoc new file mode 100644 index 000000000..1a7ab8979 --- /dev/null +++ b/modules/sql/pages/get-started/index.adoc @@ -0,0 +1,3 @@ += Get Started with Redpanda SQL +:description: Get started with Redpanda SQL, a column-oriented OLAP query engine built into Redpanda Cloud that lets you query streaming topics using standard SQL. +:page-layout: index From 6caf890a183b0cd5550dd793cd443da0f26ad3b5 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Tue, 5 May 2026 11:46:10 -0700 Subject: [PATCH 04/14] Remove create kafka source --- .../sql/pages/get-started/sql-quickstart.adoc | 96 ++++++++----------- 1 file changed, 41 insertions(+), 55 deletions(-) diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 20c706da4..5e05726e3 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -14,35 +14,6 @@ Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda BYOC. It // From PRD: SCRAM auth preserved, connection string available in Cloud Console and API response. // Confirm with engineering what SCRAM credentials does the user use - superuser auto-created by Control Plane? -== Get connection details - -After Redpanda SQL is provisioned on your cluster: - -. In the Redpanda Cloud Console, go to your cluster and open the *SQL* tab. -. Copy the connection string. - -== Connect to Redpanda SQL - -Use `psql` to connect to the SQL engine. Paste the connection string you copied from the Console: - -// TODO: Replace with actual connection string format once confirmed with engineering. -[,bash] ----- -psql "" ----- - -On a successful connection, you should see output similar to: - -// TODO: Verify current psql banner text. -[.no-copy] ----- -psql (14.x, server 16.0) -SSL connection (protocol: TLSv1.3) -Type "help" for help. - -=> ----- - [#optional-produce-sample-data] == (Optional) Produce sample data @@ -64,7 +35,6 @@ rpk topic create orders . Save the following Protobuf schema as `order.proto`: + -// TODO: Confirm the GA-supported schema format(s) with engineering. If JSON Schema is supported at GA, consider switching this example to JSON for simpler UX. [,proto] ---- syntax = "proto3"; @@ -98,36 +68,62 @@ rpk topic produce orders --schema-id=topic <" ---- -The SQL engine fetches the topic's schema from Schema Registry automatically. +On a successful connection, you should see output similar to: -Redpanda SQL supports Protobuf, Avro, and JSON schemas registered in Schema Registry. The supported data types at GA are primitive scalar types (`INT`, `BIGINT`, `FLOAT`, `BOOLEAN`, `VARCHAR`, `TIMESTAMP`, and others) and `Struct`. +// TODO: Verify current psql banner text. +[.no-copy] +---- +psql (14.x, server 16.0) +SSL connection (protocol: TLSv1.3) +Type "help" for help. -NOTE: Nested JSON objects may be surfaced as `VARCHAR` containing the raw JSON string rather than as a navigable `Struct` column. The behavior depends on the serialization format: Avro and Protobuf schemas define field types explicitly and map to `Struct` where supported. JSON Schema fields typed as `object` may fall back to `VARCHAR`. +=> +---- + +== Query a Redpanda topic + +When you enable Redpanda SQL, the engine automatically creates a Redpanda catalog named `default_redpanda_connection` that connects to your cluster. + +To query a Redpanda topic as a SQL table, define a table against the topic with `CREATE TABLE`. The following example uses the `orders` topic from the previous section. Replace `orders` with the name of your topic, and `orders-value` with the Schema Registry subject that holds the topic's value schema: + +[,sql] +---- +CREATE TABLE default_redpanda_connection=>orders WITH ( + topic = 'orders', + schema_subject = 'orders-value' +); +---- + +Redpanda SQL reads the registered Protobuf schema from Schema Registry and maps each top-level field to a SQL column. == Run queries -After you create the source, query your topic data with standard SQL. The following examples use the `orders` schema from the optional sample data section. If you're using your own topic, substitute the topic name and column names below. +After you create the table, query your topic data with standard SQL. The following examples use the `orders` schema from the optional sample data section. If you're using your own topic, substitute the table name and column names. View a sample of records: [,sql] ---- -SELECT * FROM orders LIMIT 10; +SELECT * FROM default_redpanda_connection=>orders LIMIT 10; ---- Count orders by status: @@ -135,7 +131,7 @@ Count orders by status: [,sql] ---- SELECT status, COUNT(*) AS total_orders -FROM orders +FROM default_redpanda_connection=>orders GROUP BY status; ---- @@ -144,22 +140,12 @@ Find the largest orders: [,sql] ---- SELECT order_id, customer, product, amount -FROM orders +FROM default_redpanda_connection=>orders WHERE amount > 10000 ORDER BY amount DESC LIMIT 20; ---- -== Inspect your SQL cluster - -Redpanda SQL provides built-in commands to inspect the state of your SQL cluster: - -[,sql] ----- -SHOW NODES; -- List SQL compute nodes and their status -SHOW QUERIES; -- List currently running queries ----- - == Next steps * xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. From 469c5e8ca00fd1f0de659fcd308f59b727d022b5 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Tue, 5 May 2026 11:46:24 -0700 Subject: [PATCH 05/14] Move show/inspect statements to deploy doc --- modules/sql/pages/get-started/deploy-sql-cluster.adoc | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index 76f5494ba..74ccf3404 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -123,6 +123,16 @@ After you enable Redpanda SQL, provisioning may take several minutes. To verify // TODO: Confirm with engineering if there are any other specific status indicators that users can see in the Console when SQL is running. Is the SQL tab only visible when SQL is enabled? +== Inspect your SQL cluster + +Redpanda SQL provides built-in commands to inspect the state of your SQL cluster: + +[,sql] +---- +SHOW NODES; -- List SQL compute nodes and their status +SHOW QUERIES; -- List currently running queries +---- + == Disable Redpanda SQL [WARNING] From 61bd49e235a82bc6db14f7ecbda0bcaa144137e1 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Thu, 7 May 2026 19:12:43 -0700 Subject: [PATCH 06/14] DOC-1856: Apply 9-node scale cap, admin framing, and GRANT mini-step Updates the Enable Redpanda SQL page so the horizontal-scaling range is 1 to 9 nodes (down from 1 to 12) per the PM SME 2026-05-07. Adds a TODO on the Disable Redpanda SQL section listing the specific behaviors to confirm with engineering before publication: what state is purged, what is deleted from object storage, the effect on topic and Schema Registry data, in-flight query error behavior, and re-enable semantics. Updates the quickstart so it explicitly targets an admin who can view the SQL connection details in the Cloud Console, and adds an optional admin-run mini-section showing how to grant a non-admin user SELECT access on a Redpanda SQL table. Includes a TODO referencing qa-questions.md #21 to refine the role/grant example once engineering confirms whether the SQL GRANT statement alone enforces per-user table access or whether an additional ACL/Kafka-side step is required. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../pages/get-started/deploy-sql-cluster.adoc | 17 +++++++++-- .../sql/pages/get-started/sql-quickstart.adoc | 28 +++++++++++++++++++ 2 files changed, 42 insertions(+), 3 deletions(-) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index 74ccf3404..82e8a5005 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -88,7 +88,7 @@ When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. == Scale Redpanda SQL -Redpanda SQL supports horizontal scaling from 1 to 12 nodes per cluster. Scaling to 0 is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. +Redpanda SQL supports horizontal scaling from 1 to 9 nodes per cluster. Scaling to 0 is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. [tabs] ===== @@ -97,13 +97,13 @@ Cloud Console:: -- . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. . Go to your BYOC cluster and open the *SQL* tab. -. Set the node count to a value between 1 and 12, then save. +. Set the node count to a value between 1 and 9, then save. -- Cloud API:: + -- -Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with the new replica count. Replace `{cluster.id}` with your cluster ID and `` with a value between 1 and 12: +Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with the new replica count. Replace `{cluster.id}` with your cluster ID and `` with a value between 1 and 9: [,bash] ---- @@ -135,6 +135,17 @@ SHOW QUERIES; -- List currently running queries == Disable Redpanda SQL +// TODO: Confirm with engineering exactly what happens when Redpanda SQL is +// disabled and document it precisely. Specifically: +// - What state is purged (catalog metadata, table mappings, role/grant +// data, query history, cached query results)? +// - What is deleted from object storage (Iceberg-translated data for +// Iceberg-enabled topics, internal SQL engine state)? +// - Are Redpanda topic data and Schema Registry subjects affected? +// - What error / status do clients see for in-flight queries? +// - If SQL is re-enabled later, is any state restored, or is the engine +// provisioned fresh? + [WARNING] ==== Disabling Redpanda SQL purges the stored catalog state for the SQL engine and deletes its data from object storage. In-flight queries fail when SQL is disabled. To temporarily stop SQL compute without losing state, <> instead. diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 5e05726e3..12531403a 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -4,9 +4,12 @@ Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda BYOC. It lets you query streaming data in your Redpanda topics with standard SQL, without building ETL pipelines or deploying a separate analytics system. In this quickstart, you connect with `psql` and run your first query against a Redpanda topic. +This quickstart is written for an admin who can view the SQL connection details in the Cloud Console. + == Prerequisites * A Redpanda BYOC cluster on AWS with Redpanda SQL enabled. See xref:sql:get-started/deploy-sql-cluster.adoc[]. +* Admin access to the cluster in the Redpanda Cloud Console (required to view SQL connection details). * A Redpanda topic with a schema registered in Schema Registry. If you don't have one, follow the optional <> section below to create a sample `orders` topic. * https://www.postgresql.org/download/[`psql`^] (PostgreSQL client) installed on your local machine. @@ -146,6 +149,31 @@ ORDER BY amount DESC LIMIT 20; ---- +== (Optional) Grant access to a non-admin user + +Redpanda SQL is deny-all by default. All queries to Redpanda SQL run under a single super-user SASL credential associated with `default_redpanda_connection`, but the engine enforces per-user table access through Redpanda SQL `GRANT` statements. To let a non-admin user query a table, the admin creates a role and grants access: + +// TODO: Confirm the exact GRANT mechanism with engineering. Open question: +// does the SQL `GRANT` statement alone enforce per-user table access, or is +// an additional ACL / Kafka-side step required? See qa-questions.md #21. +// Update the example below once confirmed. + +. Create a role for the user: ++ +[,sql] +---- +CREATE ROLE analyst LOGIN PASSWORD ''; +---- + +. Grant `SELECT` on the table to the role: ++ +[,sql] +---- +GRANT SELECT ON TABLE default_redpanda_connection=>orders TO analyst; +---- + +The non-admin user can now connect to Redpanda SQL with their credentials and run `SELECT` against the `orders` table. + == Next steps * xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. From 76d113a754c0292eecb7452576551bdbeed18dd7 Mon Sep 17 00:00:00 2001 From: Kat Batuigas <36839689+kbatuigas@users.noreply.github.com> Date: Mon, 11 May 2026 14:25:16 -0700 Subject: [PATCH 07/14] Update modules/sql/pages/get-started/sql-quickstart.adoc Co-authored-by: Andrew Hsu --- modules/sql/pages/get-started/sql-quickstart.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 12531403a..d82c2fa2e 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -110,7 +110,7 @@ To query a Redpanda topic as a SQL table, define a table against the topic with [,sql] ---- -CREATE TABLE default_redpanda_connection=>orders WITH ( +CREATE TABLE default_redpanda_catalog=>orders WITH ( topic = 'orders', schema_subject = 'orders-value' ); From fa463e1fb4efeb9c1d81cc4d274665c7f8b9aaca Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 13 May 2026 17:57:46 -0700 Subject: [PATCH 08/14] Change quickstart path to use UI to produce instead of rpk --- .../sql/pages/get-started/sql-quickstart.adoc | 95 +++++++++++-------- 1 file changed, 54 insertions(+), 41 deletions(-) diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index d82c2fa2e..075f01ffa 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -11,6 +11,7 @@ This quickstart is written for an admin who can view the SQL connection details * A Redpanda BYOC cluster on AWS with Redpanda SQL enabled. See xref:sql:get-started/deploy-sql-cluster.adoc[]. * Admin access to the cluster in the Redpanda Cloud Console (required to view SQL connection details). * A Redpanda topic with a schema registered in Schema Registry. If you don't have one, follow the optional <> section below to create a sample `orders` topic. +* xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6] or later installed on your local machine to generate an authentication token. * https://www.postgresql.org/download/[`psql`^] (PostgreSQL client) installed on your local machine. // TODO: Verify the exact connection string format and where users get credentials. @@ -27,16 +28,17 @@ Skip this section if you already have a Redpanda topic with a schema registered If you don't have a schema-registered topic to query yet, follow these steps to create an `orders` topic with a small set of sample records. Redpanda SQL reads the topic's schema from Schema Registry to map fields to SQL columns, so the topic must have a registered schema before you can query it. -This section uses xref:reference:rpk/index.adoc[`rpk`], which you can install by following xref:get-started:rpk-install.adoc[]. You also need a user with permissions to create topics, register schemas, and produce records. +You also need permissions to create topics, register schemas, and produce records. -. Create a topic: +. https://cloud.redpanda.com/[Log in to Redpanda Cloud^] and select your cluster. +. On the *Topics* page, click *Create Topic*. Name the topic `orders` and create it with default settings. +. On the *Schema Registry* page, click *Create new schema*. +. Create a new schema with the following: + -[,bash] ----- -rpk topic create orders ----- - -. Save the following Protobuf schema as `order.proto`: +* *Strategy*: Topic +* *Topic name*: orders +* *Schema applies to*: Value +* *Schema definition*: Select Protobuf and paste the following schema definition: + [,proto] ---- @@ -51,43 +53,52 @@ message Order { } ---- -. Register the schema against the topic's value subject: +// TODO: Verify exact steps to produce records in UI +. Return to the *Topics* page and select the `orders` topic. Produce a few sample records: + [,bash] ---- -rpk registry schema create orders-value --schema order.proto +{"order_id": 1, "customer": "alice", "product": "keyboard", "amount": 7500, "status": "completed"} +---- ++ +[,bash] +---- +{"order_id": 2, "customer": "bob", "product": "monitor", "amount": 32000, "status": "shipped"} +---- ++ +[,bash] +---- +{"order_id": 3, "customer": "carol", "product": "mouse", "amount": 4500, "status": "pending"} ---- - -. Produce a few sample records. The `--schema-id=topic` flag tells `rpk` to use the topic name strategy to look up the schema you just registered: + [,bash] ---- -rpk topic produce orders --schema-id=topic <" +psql "host= port=5432 dbname=oxla user=ignored password=$rpsql_token options='-c auth_method=bearer' sslmode=require" ---- On a successful connection, you should see output similar to: @@ -95,8 +106,8 @@ On a successful connection, you should see output similar to: // TODO: Verify current psql banner text. [.no-copy] ---- -psql (14.x, server 16.0) -SSL connection (protocol: TLSv1.3) +psql (17.8 (Homebrew), server 16.0 (oxla version: 1.0.0, build: af2dffb-Release-x86_64-GNU, asio)) +SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off, ALPN: none) Type "help" for help. => @@ -104,7 +115,7 @@ Type "help" for help. == Query a Redpanda topic -When you enable Redpanda SQL, the engine automatically creates a Redpanda catalog named `default_redpanda_connection` that connects to your cluster. +When you enable Redpanda SQL, the engine automatically creates a Redpanda catalog named `default_redpanda_catalog` that connects to your cluster. To query a Redpanda topic as a SQL table, define a table against the topic with `CREATE TABLE`. The following example uses the `orders` topic from the previous section. Replace `orders` with the name of your topic, and `orders-value` with the Schema Registry subject that holds the topic's value schema: @@ -126,7 +137,7 @@ View a sample of records: [,sql] ---- -SELECT * FROM default_redpanda_connection=>orders LIMIT 10; +SELECT * FROM default_redpanda_catalog=>orders LIMIT 10; ---- Count orders by status: @@ -134,7 +145,7 @@ Count orders by status: [,sql] ---- SELECT status, COUNT(*) AS total_orders -FROM default_redpanda_connection=>orders +FROM default_redpanda_catalog=>orders GROUP BY status; ---- @@ -143,7 +154,7 @@ Find the largest orders: [,sql] ---- SELECT order_id, customer, product, amount -FROM default_redpanda_connection=>orders +FROM default_redpanda_catalog=>orders WHERE amount > 10000 ORDER BY amount DESC LIMIT 20; @@ -151,12 +162,14 @@ LIMIT 20; == (Optional) Grant access to a non-admin user -Redpanda SQL is deny-all by default. All queries to Redpanda SQL run under a single super-user SASL credential associated with `default_redpanda_connection`, but the engine enforces per-user table access through Redpanda SQL `GRANT` statements. To let a non-admin user query a table, the admin creates a role and grants access: +Redpanda SQL is deny-all by default. All queries to Redpanda SQL run under a single super-user SASL credential associated with `default_redpanda_catalog`, but the engine enforces per-user table access through Redpanda SQL `GRANT` statements. To let a non-admin user query a table, the admin creates a role and grants access: -// TODO: Confirm the exact GRANT mechanism with engineering. Open question: -// does the SQL `GRANT` statement alone enforce per-user table access, or is -// an additional ACL / Kafka-side step required? See qa-questions.md #21. -// Update the example below once confirmed. +// TODO: Confirm the exact GRANT mechanism with engineering. Open questions: +// - Does access to SQL engine also require an accout in Redpanda Cloud (user +// or service account), with the new SQL roles? +// - Does the SQL `GRANT` statement alone enforce per-user table access, or is +// an additional ACL / Kafka-side step required? +// Update the steps and example below once confirmed. . Create a role for the user: + @@ -169,7 +182,7 @@ CREATE ROLE analyst LOGIN PASSWORD ''; + [,sql] ---- -GRANT SELECT ON TABLE default_redpanda_connection=>orders TO analyst; +GRANT SELECT ON TABLE default_redpanda_catalog=>orders TO analyst; ---- The non-admin user can now connect to Redpanda SQL with their credentials and run `SELECT` against the `orders` table. From 6eb3a667b44f839db99bea4246e57eac1cf4e57c Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 13 May 2026 18:39:17 -0700 Subject: [PATCH 09/14] Add deployment/connection UI updates --- .../pages/get-started/deploy-sql-cluster.adoc | 63 +++++++++---------- 1 file changed, 28 insertions(+), 35 deletions(-) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index 82e8a5005..630c413eb 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -6,10 +6,10 @@ Enable Redpanda SQL on a BYOC cluster to give your team the ability to query str == Prerequisites -To enable Redpanda SQL, you need: +To enable Redpanda SQL engine, you need: * Admin permissions in your Redpanda Cloud organization. -* For the Cloud API path, a valid bearer token for the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. See link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. +* If using the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API] to enable SQL, a valid bearer token for the API. See link:/api/doc/cloud-controlplane/authentication[Authenticate to the Cloud API]. == Enable Redpanda SQL @@ -25,6 +25,10 @@ Cloud Console:: . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. . Start creating a new BYOC cluster on AWS. For details and prerequisites, see xref:get-started:cluster-types/byoc/aws/create-byoc-cluster-aws.adoc[]. . In the cluster creation form, select the option to enable SQL. +// TODO: Confirm guidance to provide on selecting number of nodes +. Choose the number of SQL nodes to deploy. ++ +The minimum is one node to enable SQL. You can scale up (maximum nine nodes) or down later as needed, but the cluster must have at least one SQL node to run the engine. . Complete the remaining cluster configuration and deploy. -- @@ -32,6 +36,8 @@ Cloud API:: + -- . Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. +// TODO: confirm field name change to rpsql +// Is selecting the number of nodes available with this endpoint? . Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[`POST /v1/clusters`] request with `oxla.enabled` set to `true` in the cluster spec: + [,bash] @@ -55,7 +61,7 @@ curl -X POST "https://api.redpanda.com/v1/clusters" \ }' ---- + -For the full request body and field reference, see the link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[Create cluster API]. +For the full request body and field reference, see the link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[Create Cluster API]. . The request returns the ID of a long-running operation. Poll the link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] endpoint until the operation completes. -- ===== @@ -64,6 +70,8 @@ For the full request body and field reference, see the link:/api/doc/cloud-contr To enable, scale, or disable SQL on an existing cluster, you also need the cluster ID, which you can find in the *Details* section of the cluster overview in the Cloud Console. +// TODO: Confirm UI functionality + . Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. . Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request, replacing `{cluster.id}` with your cluster ID: + @@ -90,19 +98,8 @@ When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. Redpanda SQL supports horizontal scaling from 1 to 9 nodes per cluster. Scaling to 0 is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. -[tabs] -===== -Cloud Console:: -+ --- -. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to your BYOC cluster and open the *SQL* tab. -. Set the node count to a value between 1 and 9, then save. --- +// TODO: Confirm UI functionality -Cloud API:: -+ --- Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with the new replica count. Replace `{cluster.id}` with your cluster ID and `` with a value between 1 and 9: [,bash] @@ -114,14 +111,24 @@ curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ ---- The request returns the ID of a long-running operation. Poll link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] until the operation completes. --- -===== == Verify the SQL engine is running -After you enable Redpanda SQL, provisioning may take several minutes. To verify the SQL engine is running: +After you enable Redpanda SQL, the cluster overview page in the Cloud console shows the *SQL* tab. The *Details* pane also displays the number of SQL nodes deployed with the cluster. Provisioning may take several minutes. + +To verify the SQL engine is running, use the connection details on the *SQL* tab to connect with a PostgreSQL client, such as `psql`. -// TODO: Confirm with engineering if there are any other specific status indicators that users can see in the Console when SQL is running. Is the SQL tab only visible when SQL is enabled? +The following shows how to connect using a bearer token. The `rpk cloud auth token` command (xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6+] required) retrieves a temporary authentication token for the SQL engine. If your `rpk` session is not authenticated to Redpanda Cloud, the command prompts you to first log in on your browser: + +// TODO: Confirm behavior when SQL is enabled but not yet fully provisioned. +// Does the `psql` connection attempt time out, or return an error message +//indicating the engine is not ready? +[,bash] +---- +rpsql_token=$(rpk cloud auth token) + +psql "host= port=5432 dbname=oxla user=ignored password=$rpsql_token options='-c auth_method=bearer' sslmode=require" +---- == Inspect your SQL cluster @@ -130,6 +137,7 @@ Redpanda SQL provides built-in commands to inspect the state of your SQL cluster [,sql] ---- SHOW NODES; -- List SQL compute nodes and their status +SHOW REDPANDA TABLES; -- List SQL tables mapped to Redpanda topics SHOW QUERIES; -- List currently running queries ---- @@ -148,22 +156,9 @@ SHOW QUERIES; -- List currently running queries [WARNING] ==== -Disabling Redpanda SQL purges the stored catalog state for the SQL engine and deletes its data from object storage. In-flight queries fail when SQL is disabled. To temporarily stop SQL compute without losing state, <> instead. +Disabling Redpanda SQL purges the stored catalog state for the SQL engine and deletes its data from object storage. In-flight queries fail when SQL is disabled. ==== -[tabs] -===== -Cloud Console:: -+ --- -. Log in to https://cloud.redpanda.com[Redpanda Cloud^]. -. Go to your BYOC cluster and open the *SQL* tab. -. Click *Remove* and confirm. --- - -Cloud API:: -+ --- Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with `oxla.enabled` set to `false`. Replace `{cluster.id}` with your cluster ID: [,bash] @@ -175,8 +170,6 @@ curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ ---- The request returns the ID of a long-running operation. Poll link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] until the operation completes. --- -===== == Next steps From 50d71318f3bb8a2e4e5d7fc92bdfcd1dd5228f9a Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 18 May 2026 14:35:21 -0700 Subject: [PATCH 10/14] Apply suggestions from SME --- .../pages/get-started/deploy-sql-cluster.adoc | 34 +++++------ .../sql/pages/get-started/sql-quickstart.adoc | 59 +++++++++++-------- 2 files changed, 50 insertions(+), 43 deletions(-) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index 630c413eb..6f59ae118 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -2,7 +2,7 @@ :description: Enable the Redpanda SQL engine on a BYOC cluster so that users can query streaming data with standard PostgreSQL syntax. :page-topic-type: how-to -Enable Redpanda SQL on a BYOC cluster to give your team the ability to query streaming data in Redpanda topics using standard PostgreSQL syntax. +Enable Redpanda SQL on a BYOC cluster to give your team the ability to query streaming data in Redpanda topics using standard PostgreSQL syntax. Iceberg-enabled topics can also be queried alongside their Iceberg-translated history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics] for that workflow. == Prerequisites @@ -96,7 +96,7 @@ When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. == Scale Redpanda SQL -Redpanda SQL supports horizontal scaling from 1 to 9 nodes per cluster. Scaling to 0 is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. +Redpanda SQL supports horizontal scaling from one to nine nodes per cluster. Scaling to zero nodes is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. // TODO: Confirm UI functionality @@ -114,17 +114,18 @@ The request returns the ID of a long-running operation. Poll link:/api/doc/cloud == Verify the SQL engine is running -After you enable Redpanda SQL, the cluster overview page in the Cloud console shows the *SQL* tab. The *Details* pane also displays the number of SQL nodes deployed with the cluster. Provisioning may take several minutes. +After you enable Redpanda SQL, the cluster overview page in the Cloud Console shows the *SQL* tab and the *Details* pane displays the number of SQL nodes deployed with the cluster. -To verify the SQL engine is running, use the connection details on the *SQL* tab to connect with a PostgreSQL client, such as `psql`. +The *SQL* tab appears as soon as you enable SQL, but you can't connect until the engine is fully provisioned. Provisioning can take up to 30 minutes. Wait for the node-ready status indicator on the overview page to show the engine is ready. For the API flow, poll the long-running operation until it returns `STATE_COMPLETED`. -The following shows how to connect using a bearer token. The `rpk cloud auth token` command (xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6+] required) retrieves a temporary authentication token for the SQL engine. If your `rpk` session is not authenticated to Redpanda Cloud, the command prompts you to first log in on your browser: +To verify the SQL engine is running, use the connection details on the *SQL* tab to connect with a PostgreSQL client, such as `psql` (v16 or later required). + +The following shows how to connect using a bearer token. Log in to Redpanda Cloud with `rpk`, then retrieve a temporary authentication token for the SQL engine with `rpk cloud auth token` (xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6+] required): -// TODO: Confirm behavior when SQL is enabled but not yet fully provisioned. -// Does the `psql` connection attempt time out, or return an error message -//indicating the engine is not ready? [,bash] ---- +rpk cloud login + rpsql_token=$(rpk cloud auth token) psql "host= port=5432 dbname=oxla user=ignored password=$rpsql_token options='-c auth_method=bearer' sslmode=require" @@ -143,22 +144,15 @@ SHOW QUERIES; -- List currently running queries == Disable Redpanda SQL -// TODO: Confirm with engineering exactly what happens when Redpanda SQL is -// disabled and document it precisely. Specifically: -// - What state is purged (catalog metadata, table mappings, role/grant -// data, query history, cached query results)? -// - What is deleted from object storage (Iceberg-translated data for -// Iceberg-enabled topics, internal SQL engine state)? -// - Are Redpanda topic data and Schema Registry subjects affected? -// - What error / status do clients see for in-flight queries? -// - If SQL is re-enabled later, is any state restored, or is the engine -// provisioned fresh? - [WARNING] ==== -Disabling Redpanda SQL purges the stored catalog state for the SQL engine and deletes its data from object storage. In-flight queries fail when SQL is disabled. +Disabling Redpanda SQL purges the stored catalog state for the SQL engine (catalog metadata, table mappings, and role/grant data) and deletes its data from object storage, including Iceberg-translated data for Iceberg-enabled topics. In-flight queries fail when SQL is disabled. ==== +Redpanda topic data and Schema Registry subjects are not affected. The Redpanda cluster itself continues to run normally; only the SQL engine and its associated state are removed. + +Re-enabling SQL on the same cluster provisions a fresh engine: no prior catalog state, table mappings, or grants are restored. You must re-create catalogs, tables, and grants after re-enabling. + Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with `oxla.enabled` set to `false`. Replace `{cluster.id}` with your cluster ID: [,bash] diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 075f01ffa..8e96a9f23 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -1,23 +1,29 @@ = Redpanda SQL quickstart :description: Connect to Redpanda SQL on a BYOC cluster and run your first query on streaming data. -:page-topic-type: guide +:page-topic-type: tutorial +:personas: platform_admin +:learning-objective-1: Enable Redpanda SQL on a BYOC cluster +:learning-objective-2: Connect to Redpanda SQL using psql and a bearer token +:learning-objective-3: Query a Redpanda topic with SQL -Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda BYOC. It lets you query streaming data in your Redpanda topics with standard SQL, without building ETL pipelines or deploying a separate analytics system. In this quickstart, you connect with `psql` and run your first query against a Redpanda topic. +Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda Bring Your Own Cloud (BYOC). It lets you query streaming data in your Redpanda topics with standard SQL, without building ETL pipelines or deploying a separate analytics system. In this quickstart, you connect with `psql` and run your first query against a Redpanda topic. -This quickstart is written for an admin who can view the SQL connection details in the Cloud Console. +For Iceberg-enabled topics, Redpanda SQL can also query the full Iceberg-translated history alongside live records. For that workflow, see xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]. + +After completing this quickstart, you will be able to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} == Prerequisites * A Redpanda BYOC cluster on AWS with Redpanda SQL enabled. See xref:sql:get-started/deploy-sql-cluster.adoc[]. -* Admin access to the cluster in the Redpanda Cloud Console (required to view SQL connection details). +* Admin access to your cluster in the Redpanda Cloud Console, or a role with the *SQL: Manage* permission. You need one of these to view SQL connection details and to create catalogs, tables, and grants in the SQL engine. For more information on authorization in Redpanda Cloud, see xref:security:authorization/rbac/index.adoc[]. * A Redpanda topic with a schema registered in Schema Registry. If you don't have one, follow the optional <> section below to create a sample `orders` topic. * xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6] or later installed on your local machine to generate an authentication token. * https://www.postgresql.org/download/[`psql`^] (PostgreSQL client) installed on your local machine. -// TODO: Verify the exact connection string format and where users get credentials. -// From PRD: SCRAM auth preserved, connection string available in Cloud Console and API response. -// Confirm with engineering what SCRAM credentials does the user use - superuser auto-created by Control Plane? - [#optional-produce-sample-data] == (Optional) Produce sample data @@ -87,7 +93,14 @@ When you continue to the next section, use `orders` as the topic name when you d SQL connection details are available on your cluster's *SQL* tab in the https://cloud.redpanda.com/[Cloud console]. To connect using `psql`: -. Get an authentication token using `rpk`. If prompted, log in to Redpanda Cloud in your browser: +. Log in to Redpanda Cloud with `rpk`. This opens a browser window for SSO sign-in: ++ +[,bash] +---- +rpk cloud login +---- + +. Retrieve a temporary authentication token for the SQL engine: + [,bash] ---- @@ -131,7 +144,7 @@ Redpanda SQL reads the registered Protobuf schema from Schema Registry and maps == Run queries -After you create the table, query your topic data with standard SQL. The following examples use the `orders` schema from the optional sample data section. If you're using your own topic, substitute the table name and column names. +After you create the table, query the topic with standard SQL. The following examples use the `orders` schema from the optional sample data section. If you're using your own topic, substitute the table name and column names. View a sample of records: @@ -162,30 +175,30 @@ LIMIT 20; == (Optional) Grant access to a non-admin user -Redpanda SQL is deny-all by default. All queries to Redpanda SQL run under a single super-user SASL credential associated with `default_redpanda_catalog`, but the engine enforces per-user table access through Redpanda SQL `GRANT` statements. To let a non-admin user query a table, the admin creates a role and grants access: +Redpanda Cloud's data-plane RBAC controls Redpanda SQL access through two role permissions: + +* *SQL: Manage*: superuser access to the SQL engine. A user with this role can read all topics, create catalogs and tables, and grant access to other users. +* *SQL: Access*: regular user access. A user with this role can connect to the SQL engine but has no access to any catalog or table until a SQL: Manage user grants it. -// TODO: Confirm the exact GRANT mechanism with engineering. Open questions: -// - Does access to SQL engine also require an accout in Redpanda Cloud (user -// or service account), with the new SQL roles? -// - Does the SQL `GRANT` statement alone enforce per-user table access, or is -// an additional ACL / Kafka-side step required? -// Update the steps and example below once confirmed. +When you assign one of these roles to a user in Redpanda Cloud, the cluster provisions a corresponding user in the SQL engine. No manual `CREATE USER` is required. A SQL: Manage user then uses standard SQL `GRANT` statements to give the user access to specific catalogs or tables. Wildcard patterns are supported. -. Create a role for the user: +. In Redpanda Cloud, assign a role with the *SQL: Access* permission to the user. Roles are managed in *Organization IAM > Roles*; SQL permissions are under the *Data Plane* tab when you create or edit a role. See xref:security:authorization/rbac/rbac_dp.adoc[]. + +. As a SQL: Manage user, grant `SELECT` on a specific table. The user identifier is the email on the user's Redpanda Cloud account: + [,sql] ---- -CREATE ROLE analyst LOGIN PASSWORD ''; +GRANT SELECT ON TABLE default_redpanda_catalog=>orders TO "alice@example.com"; ---- - -. Grant `SELECT` on the table to the role: ++ +Or grant `SELECT` on multiple tables that match a pattern: + [,sql] ---- -GRANT SELECT ON TABLE default_redpanda_catalog=>orders TO analyst; +GRANT SELECT ON TABLE default_redpanda_catalog=>orders_* TO "alice@example.com"; ---- -The non-admin user can now connect to Redpanda SQL with their credentials and run `SELECT` against the `orders` table. +The user can now connect to Redpanda SQL and run `SELECT` against the tables they've been granted. == Next steps From d525341b5c84b9c2b8cb05bc46e905903359c705 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 18 May 2026 16:30:51 -0700 Subject: [PATCH 11/14] Review pass --- .../pages/get-started/deploy-sql-cluster.adoc | 21 ++++++-- .../sql/pages/get-started/sql-quickstart.adoc | 52 +++++++++---------- 2 files changed, 42 insertions(+), 31 deletions(-) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index 6f59ae118..c4ba6b7f7 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -1,8 +1,18 @@ = Enable Redpanda SQL on a BYOC cluster -:description: Enable the Redpanda SQL engine on a BYOC cluster so that users can query streaming data with standard PostgreSQL syntax. +:description: Enable the Redpanda SQL engine on a BYOC cluster so you can query streaming data with standard PostgreSQL syntax. :page-topic-type: how-to +:personas: platform_admin, data_engineer +:learning-objective-1: Enable Redpanda SQL on a new or existing BYOC cluster +:learning-objective-2: Scale or disable the SQL engine using the Cloud API +:learning-objective-3: Verify that the SQL engine is running and ready to accept connections -Enable Redpanda SQL on a BYOC cluster to give your team the ability to query streaming data in Redpanda topics using standard PostgreSQL syntax. Iceberg-enabled topics can also be queried alongside their Iceberg-translated history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics] for that workflow. +Enable Redpanda SQL on a BYOC cluster so you can query streaming data in Redpanda topics using standard PostgreSQL syntax. Iceberg-enabled topics can also be queried alongside their Iceberg-translated history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics] for that workflow. + +After reading this page, you will be able to: + +* [ ] {learning-objective-1} +* [ ] {learning-objective-2} +* [ ] {learning-objective-3} == Prerequisites @@ -35,7 +45,7 @@ The minimum is one node to enable SQL. You can scale up (maximum nine nodes) or Cloud API:: + -- -. Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. +. Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the Cloud API]. // TODO: confirm field name change to rpsql // Is selecting the number of nodes available with this endpoint? . Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[`POST /v1/clusters`] request with `oxla.enabled` set to `true` in the cluster spec: @@ -72,7 +82,7 @@ To enable, scale, or disable SQL on an existing cluster, you also need the clust // TODO: Confirm UI functionality -. Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the API]. +. Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the Cloud API]. . Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request, replacing `{cluster.id}` with your cluster ID: + [,bash] @@ -96,7 +106,7 @@ When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. == Scale Redpanda SQL -Redpanda SQL supports horizontal scaling from one to nine nodes per cluster. Scaling to zero nodes is not supported. To remove Redpanda SQL from a cluster, disable the SQL engine instead. +Redpanda SQL supports horizontal scaling from one to nine nodes per cluster. You cannot scale to zero nodes. To remove Redpanda SQL from a cluster, disable the SQL engine instead. // TODO: Confirm UI functionality @@ -168,3 +178,4 @@ The request returns the ID of a long-running operation. Poll link:/api/doc/cloud == Next steps * xref:sql:get-started/sql-quickstart.adoc[Quickstart]: Connect to Redpanda SQL with `psql` and run your first query. +* xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 8e96a9f23..0b475625d 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -1,20 +1,18 @@ -= Redpanda SQL quickstart += Redpanda SQL Quickstart :description: Connect to Redpanda SQL on a BYOC cluster and run your first query on streaming data. :page-topic-type: tutorial -:personas: platform_admin -:learning-objective-1: Enable Redpanda SQL on a BYOC cluster -:learning-objective-2: Connect to Redpanda SQL using psql and a bearer token -:learning-objective-3: Query a Redpanda topic with SQL +:personas: streaming_developer, data_engineer, platform_admin +:learning-objective-1: Connect to Redpanda SQL using psql and a bearer token +:learning-objective-2: Query a Redpanda topic with SQL Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda Bring Your Own Cloud (BYOC). It lets you query streaming data in your Redpanda topics with standard SQL, without building ETL pipelines or deploying a separate analytics system. In this quickstart, you connect with `psql` and run your first query against a Redpanda topic. For Iceberg-enabled topics, Redpanda SQL can also query the full Iceberg-translated history alongside live records. For that workflow, see xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]. -After completing this quickstart, you will be able to: +After reading this page, you will be able to: * [ ] {learning-objective-1} * [ ] {learning-objective-2} -* [ ] {learning-objective-3} == Prerequisites @@ -37,8 +35,11 @@ If you don't have a schema-registered topic to query yet, follow these steps to You also need permissions to create topics, register schemas, and produce records. . https://cloud.redpanda.com/[Log in to Redpanda Cloud^] and select your cluster. + . On the *Topics* page, click *Create Topic*. Name the topic `orders` and create it with default settings. + . On the *Schema Registry* page, click *Create new schema*. + . Create a new schema with the following: + * *Strategy*: Topic @@ -59,8 +60,8 @@ message Order { } ---- -// TODO: Verify exact steps to produce records in UI . Return to the *Topics* page and select the `orders` topic. Produce a few sample records: +// TODO: Verify exact steps to produce records in UI + [,bash] ---- @@ -114,7 +115,7 @@ rpsql_token=$(rpk cloud auth token) psql "host= port=5432 dbname=oxla user=ignored password=$rpsql_token options='-c auth_method=bearer' sslmode=require" ---- -On a successful connection, you should see output similar to: +On a successful connection, you see output similar to: // TODO: Verify current psql banner text. [.no-copy] @@ -126,12 +127,12 @@ Type "help" for help. => ---- -== Query a Redpanda topic - -When you enable Redpanda SQL, the engine automatically creates a Redpanda catalog named `default_redpanda_catalog` that connects to your cluster. +== Query topic data -To query a Redpanda topic as a SQL table, define a table against the topic with `CREATE TABLE`. The following example uses the `orders` topic from the previous section. Replace `orders` with the name of your topic, and `orders-value` with the Schema Registry subject that holds the topic's value schema: +When you enable Redpanda SQL, the engine automatically creates a Redpanda catalog named `default_redpanda_catalog` that connects to your cluster. To query a Redpanda topic, define a SQL table against the topic, then run standard SQL queries against the table. +. Define a SQL table from the topic with `CREATE TABLE`. The following example uses the `orders` topic from the optional sample data section. To use your own topic, replace `orders` with your topic name and `orders-value` with the Schema Registry subject that holds the topic's value schema. Your topic must have a schema registered in Schema Registry. ++ [,sql] ---- CREATE TABLE default_redpanda_catalog=>orders WITH ( @@ -139,31 +140,29 @@ CREATE TABLE default_redpanda_catalog=>orders WITH ( schema_subject = 'orders-value' ); ---- ++ +Redpanda SQL reads the registered schema and maps each top-level field to a SQL column. -Redpanda SQL reads the registered Protobuf schema from Schema Registry and maps each top-level field to a SQL column. - -== Run queries - -After you create the table, query the topic with standard SQL. The following examples use the `orders` schema from the optional sample data section. If you're using your own topic, substitute the table name and column names. - +. Run SQL queries against the table. These examples use the `orders` schema from the optional sample data section. ++ View a sample of records: - ++ [,sql] ---- SELECT * FROM default_redpanda_catalog=>orders LIMIT 10; ---- - ++ Count orders by status: - ++ [,sql] ---- SELECT status, COUNT(*) AS total_orders FROM default_redpanda_catalog=>orders GROUP BY status; ---- - ++ Find the largest orders: - ++ [,sql] ---- SELECT order_id, customer, product, amount @@ -202,6 +201,7 @@ The user can now connect to Redpanda SQL and run `SELECT` against the tables the == Next steps +// TODO: Uncomment once DOC-1990 and DOC-2006 merge (target pages are on those branches). +// * xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: Map a Redpanda topic to a SQL table and run analytical queries against live streaming data. +// * xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]: Run a single SQL query that spans live records and the Iceberg-translated history of a topic. * xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. -* xref:sql:connect-to-sql/language-clients/psycopg2.adoc[Connect with Python (psycopg2)]: Query Redpanda SQL programmatically. -* xref:sql:connect-to-sql/language-clients/java-jdbc.adoc[Connect with Java (JDBC)]: Integrate with Java applications. From a0b51f4e5a6887c42368c971e555d735c84e37d4 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Mon, 18 May 2026 16:59:41 -0700 Subject: [PATCH 12/14] Add wire protocol option when producing via Console --- modules/sql/pages/get-started/sql-quickstart.adoc | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 0b475625d..f20463295 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -20,7 +20,7 @@ After reading this page, you will be able to: * Admin access to your cluster in the Redpanda Cloud Console, or a role with the *SQL: Manage* permission. You need one of these to view SQL connection details and to create catalogs, tables, and grants in the SQL engine. For more information on authorization in Redpanda Cloud, see xref:security:authorization/rbac/index.adoc[]. * A Redpanda topic with a schema registered in Schema Registry. If you don't have one, follow the optional <> section below to create a sample `orders` topic. * xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6] or later installed on your local machine to generate an authentication token. -* https://www.postgresql.org/download/[`psql`^] (PostgreSQL client) installed on your local machine. +* https://www.postgresql.org/download/[`psql`^] v16 or later (PostgreSQL client) installed on your local machine. [#optional-produce-sample-data] == (Optional) Produce sample data @@ -137,11 +137,14 @@ When you enable Redpanda SQL, the engine automatically creates a Redpanda catalo ---- CREATE TABLE default_redpanda_catalog=>orders WITH ( topic = 'orders', - schema_subject = 'orders-value' + schema_subject = 'orders-value', + confluent_wire_protocol = 'false' ); ---- + Redpanda SQL reads the registered schema and maps each top-level field to a SQL column. ++ +Records produced through the Cloud Console don't carry the Confluent Schema Registry wire-format prefix, so the example sets `confluent_wire_protocol = 'false'`. If your producer client adds the wire format, set this option to `'true'` or omit it. . Run SQL queries against the table. These examples use the `orders` schema from the optional sample data section. + From 746cf368945e185ce8d6c2d3f62abf8b1825ef26 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 20 May 2026 15:40:37 -0700 Subject: [PATCH 13/14] Edits per SME input --- .../pages/get-started/deploy-sql-cluster.adoc | 94 +++++++++++++++---- .../sql/pages/get-started/sql-quickstart.adoc | 40 ++++---- 2 files changed, 92 insertions(+), 42 deletions(-) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index c4ba6b7f7..0c7fedbf8 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -1,12 +1,12 @@ -= Enable Redpanda SQL on a BYOC cluster += Enable Redpanda SQL on a BYOC Cluster :description: Enable the Redpanda SQL engine on a BYOC cluster so you can query streaming data with standard PostgreSQL syntax. :page-topic-type: how-to :personas: platform_admin, data_engineer :learning-objective-1: Enable Redpanda SQL on a new or existing BYOC cluster -:learning-objective-2: Scale or disable the SQL engine using the Cloud API +:learning-objective-2: Scale or disable the SQL engine :learning-objective-3: Verify that the SQL engine is running and ready to accept connections -Enable Redpanda SQL on a BYOC cluster so you can query streaming data in Redpanda topics using standard PostgreSQL syntax. Iceberg-enabled topics can also be queried alongside their Iceberg-translated history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics] for that workflow. +Enable Redpanda SQL on a Bring Your Own Cloud (BYOC) cluster so you can query streaming data in Redpanda topics using standard PostgreSQL syntax. Iceberg-enabled topics can also be queried alongside their Iceberg-translated history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics] for that workflow. After reading this page, you will be able to: @@ -18,6 +18,7 @@ After reading this page, you will be able to: To enable Redpanda SQL engine, you need: +* A Redpanda Cloud organization on xref:billing:billing.adoc[usage-based billing]. * Admin permissions in your Redpanda Cloud organization. * If using the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API] to enable SQL, a valid bearer token for the API. See link:/api/doc/cloud-controlplane/authentication[Authenticate to the Cloud API]. @@ -35,8 +36,9 @@ Cloud Console:: . Log in to https://cloud.redpanda.com[Redpanda Cloud^]. . Start creating a new BYOC cluster on AWS. For details and prerequisites, see xref:get-started:cluster-types/byoc/aws/create-byoc-cluster-aws.adoc[]. . In the cluster creation form, select the option to enable SQL. -// TODO: Confirm guidance to provide on selecting number of nodes -. Choose the number of SQL nodes to deploy. +// TODO: Confirm guidance to provide on selecting capacity. The node-count +// selection is expected to be replaced with RPUs +. Choose the number of SQL nodes to deploy. + The minimum is one node to enable SQL. You can scale up (maximum nine nodes) or down later as needed, but the cluster must have at least one SQL node to run the engine. . Complete the remaining cluster configuration and deploy. @@ -46,9 +48,9 @@ Cloud API:: + -- . Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the Cloud API]. -// TODO: confirm field name change to rpsql -// Is selecting the number of nodes available with this endpoint? -. Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[`POST /v1/clusters`] request with `oxla.enabled` set to `true` in the cluster spec: +// TODO: Is selecting capacity (currently node count, expected to change to RPUs) +// available with this endpoint? +. Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_createcluster[`POST /v1/clusters`] request with `rpsql.enabled` set to `true` in the cluster spec: + [,bash] ---- @@ -64,7 +66,7 @@ curl -X POST "https://api.redpanda.com/v1/clusters" \ "zones": [ ], "throughput_tier": "", "resource_group_id": "", - "oxla": { + "rpsql": { "enabled": true } } @@ -78,11 +80,25 @@ For the full request body and field reference, see the link:/api/doc/cloud-contr === On an existing cluster -To enable, scale, or disable SQL on an existing cluster, you also need the cluster ID, which you can find in the *Details* section of the cluster overview in the Cloud Console. +// TODO: The node-count selection in the Edit RP SQL dialog is expected to be +// replaced with one for RPUs -// TODO: Confirm UI functionality +[tabs] +===== +Cloud Console:: ++ +-- +. Log in to https://cloud.redpanda.com[Redpanda Cloud^] and open your cluster. +. From the left navigation, select *Dataplane settings*. +. On the *Cluster* tab, find the *RP SQL* row and click *Edit*. +. In the *Edit RP SQL* dialog, select *Enable*, set the *Number of RP SQL nodes* (1-9), and click *Save*. +-- +Cloud API:: ++ +-- . Authenticate to the link:/api/doc/cloud-controlplane/topic/topic-cloud-api-overview[Cloud API]. For details, see link:/api/doc/cloud-controlplane/authentication[Authenticate to the Cloud API]. +. Locate the cluster ID in the *Details* section of the cluster overview in the Cloud Console. . Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request, replacing `{cluster.id}` with your cluster ID: + [,bash] @@ -90,7 +106,7 @@ To enable, scale, or disable SQL on an existing cluster, you also need the clust curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ -H "Authorization: Bearer $AUTH_TOKEN" \ -H "Content-Type: application/json" \ - -d '{"oxla":{"enabled":true}}' + -d '{"rpsql":{"enabled":true}}' ---- + The request returns the ID of a long-running operation. Poll the link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] endpoint until the operation completes: @@ -103,13 +119,30 @@ curl -X GET "https://api.redpanda.com/v1/operations/{operation.id}" \ ---- + When the operation is complete, the response shows `"state": "STATE_COMPLETED"`. +-- +===== == Scale Redpanda SQL Redpanda SQL supports horizontal scaling from one to nine nodes per cluster. You cannot scale to zero nodes. To remove Redpanda SQL from a cluster, disable the SQL engine instead. -// TODO: Confirm UI functionality +// TODO: Scaling unit is changing. The node-count input (1-9) shown in the +// current UI is expected to be replaced with RPUs + +[tabs] +===== +Cloud Console:: ++ +-- +. Log in to https://cloud.redpanda.com[Redpanda Cloud^] and open your cluster. +. From the left navigation, select *Dataplane settings*. +. On the *Cluster* tab, find the *RP SQL* row and click *Edit*. +. In the *Edit RP SQL* dialog, keep *Enable* selected, update *Number of RP SQL nodes* to a value between 1 and 9, and click *Save*. +-- +Cloud API:: ++ +-- Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with the new replica count. Replace `{cluster.id}` with your cluster ID and `` with a value between 1 and 9: [,bash] @@ -117,15 +150,18 @@ Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updat curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ -H "Authorization: Bearer $AUTH_TOKEN" \ -H "Content-Type: application/json" \ - -d '{"oxla":{"replicas":}}' + -d '{"rpsql":{"replicas":}}' ---- The request returns the ID of a long-running operation. Poll link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] until the operation completes. +-- +===== == Verify the SQL engine is running After you enable Redpanda SQL, the cluster overview page in the Cloud Console shows the *SQL* tab and the *Details* pane displays the number of SQL nodes deployed with the cluster. +// TODO: Confirm UI changes for SQL engine ready indicator The *SQL* tab appears as soon as you enable SQL, but you can't connect until the engine is fully provisioned. Provisioning can take up to 30 minutes. Wait for the node-ready status indicator on the overview page to show the engine is ready. For the API flow, poll the long-running operation until it returns `STATE_COMPLETED`. To verify the SQL engine is running, use the connection details on the *SQL* tab to connect with a PostgreSQL client, such as `psql` (v16 or later required). @@ -147,9 +183,11 @@ Redpanda SQL provides built-in commands to inspect the state of your SQL cluster [,sql] ---- -SHOW NODES; -- List SQL compute nodes and their status -SHOW REDPANDA TABLES; -- List SQL tables mapped to Redpanda topics -SHOW QUERIES; -- List currently running queries +SHOW NODES; -- List SQL compute nodes and their status +SHOW REDPANDA CATALOGS; -- List Redpanda catalogs +SHOW ICEBERG CATALOGS; -- List Iceberg catalogs +SHOW REDPANDA TABLES; -- List SQL tables mapped to Redpanda topics +SHOW QUERIES; -- List currently running queries ---- == Disable Redpanda SQL @@ -163,19 +201,37 @@ Redpanda topic data and Schema Registry subjects are not affected. The Redpanda Re-enabling SQL on the same cluster provisions a fresh engine: no prior catalog state, table mappings, or grants are restored. You must re-create catalogs, tables, and grants after re-enabling. -Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with `oxla.enabled` set to `false`. Replace `{cluster.id}` with your cluster ID: +[tabs] +===== +Cloud Console:: ++ +-- +. Log in to https://cloud.redpanda.com[Redpanda Cloud^] and open your cluster. +. From the left navigation, select *Dataplane settings*. +. On the *Cluster* tab, find the *RP SQL* row and click *Edit*. +. In the *Edit RP SQL* dialog, select *Disable*, then click *Save*. +-- + +Cloud API:: ++ +-- +Make a link:/api/doc/cloud-controlplane/operation/operation-clusterservice_updatecluster[`PATCH /v1/clusters/{cluster.id}`] request with `rpsql.enabled` set to `false`. Replace `{cluster.id}` with your cluster ID: [,bash] ---- curl -X PATCH "https://api.redpanda.com/v1/clusters/{cluster.id}" \ -H "Authorization: Bearer $AUTH_TOKEN" \ -H "Content-Type: application/json" \ - -d '{"oxla":{"enabled":false}}' + -d '{"rpsql":{"enabled":false}}' ---- The request returns the ID of a long-running operation. Poll link:/api/doc/cloud-controlplane/operation/operation-operationservice_getoperation[`GET /v1/operations/{operation.id}`] until the operation completes. +-- +===== == Next steps * xref:sql:get-started/sql-quickstart.adoc[Quickstart]: Connect to Redpanda SQL with `psql` and run your first query. +// TODO: Uncomment once DOC-1999 merges (target page is on that branch). +// * xref:sql:connect-to-sql/authenticate.adoc[Authenticate to Redpanda SQL]: Connect with an OIDC bearer token, OIDC client credentials, or a SCRAM password for clients that don't support OIDC. * xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index f20463295..3ae6e2edd 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -5,9 +5,7 @@ :learning-objective-1: Connect to Redpanda SQL using psql and a bearer token :learning-objective-2: Query a Redpanda topic with SQL -Redpanda SQL is a PostgreSQL-compatible SQL engine built into Redpanda Bring Your Own Cloud (BYOC). It lets you query streaming data in your Redpanda topics with standard SQL, without building ETL pipelines or deploying a separate analytics system. In this quickstart, you connect with `psql` and run your first query against a Redpanda topic. - -For Iceberg-enabled topics, Redpanda SQL can also query the full Iceberg-translated history alongside live records. For that workflow, see xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]. +Connect to Redpanda SQL on a Bring Your Own Cloud (BYOC) cluster and run your first query against streaming data, with no ETL pipelines required. For querying topics with Iceberg-translated history, see xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]. After reading this page, you will be able to: @@ -23,7 +21,7 @@ After reading this page, you will be able to: * https://www.postgresql.org/download/[`psql`^] v16 or later (PostgreSQL client) installed on your local machine. [#optional-produce-sample-data] -== (Optional) Produce sample data +== Produce sample data [TIP] ==== @@ -129,7 +127,7 @@ Type "help" for help. == Query topic data -When you enable Redpanda SQL, the engine automatically creates a Redpanda catalog named `default_redpanda_catalog` that connects to your cluster. To query a Redpanda topic, define a SQL table against the topic, then run standard SQL queries against the table. +When you enable Redpanda SQL, a Redpanda catalog named `default_redpanda_catalog` is created automatically and points at your cluster. To query a Redpanda topic, define a SQL table against the topic, then run standard SQL queries against the table. . Define a SQL table from the topic with `CREATE TABLE`. The following example uses the `orders` topic from the optional sample data section. To use your own topic, replace `orders` with your topic name and `orders-value` with the Schema Registry subject that holds the topic's value schema. Your topic must have a schema registered in Schema Registry. + @@ -144,7 +142,7 @@ CREATE TABLE default_redpanda_catalog=>orders WITH ( + Redpanda SQL reads the registered schema and maps each top-level field to a SQL column. + -Records produced through the Cloud Console don't carry the Confluent Schema Registry wire-format prefix, so the example sets `confluent_wire_protocol = 'false'`. If your producer client adds the wire format, set this option to `'true'` or omit it. +Records produced through the Cloud Console don't carry the Schema Registry wire-format prefix, so the example sets `confluent_wire_protocol = 'false'`. If your producer client adds the wire format, set this option to `'true'` or omit it. For details on this and other CREATE TABLE options, see xref:reference:sql/sql-statements/create-table.adoc[CREATE TABLE]. . Run SQL queries against the table. These examples use the `orders` schema from the optional sample data section. + @@ -175,36 +173,32 @@ ORDER BY amount DESC LIMIT 20; ---- -== (Optional) Grant access to a non-admin user - -Redpanda Cloud's data-plane RBAC controls Redpanda SQL access through two role permissions: +== Grant access to a non-admin user -* *SQL: Manage*: superuser access to the SQL engine. A user with this role can read all topics, create catalogs and tables, and grant access to other users. -* *SQL: Access*: regular user access. A user with this role can connect to the SQL engine but has no access to any catalog or table until a SQL: Manage user grants it. +[TIP] +==== +Skip this section if you don't need to share SQL access yet. +==== -When you assign one of these roles to a user in Redpanda Cloud, the cluster provisions a corresponding user in the SQL engine. No manual `CREATE USER` is required. A SQL: Manage user then uses standard SQL `GRANT` statements to give the user access to specific catalogs or tables. Wildcard patterns are supported. +To share SQL access with a teammate or service account: -. In Redpanda Cloud, assign a role with the *SQL: Access* permission to the user. Roles are managed in *Organization IAM > Roles*; SQL permissions are under the *Data Plane* tab when you create or edit a role. See xref:security:authorization/rbac/rbac_dp.adoc[]. +. In Redpanda Cloud, assign the user a role with the *SQL: Access* permission (see xref:security:authorization/rbac/rbac_dp.adoc[Configure RBAC in the Data Plane]). Redpanda Cloud provisions a corresponding user in the SQL engine. -. As a SQL: Manage user, grant `SELECT` on a specific table. The user identifier is the email on the user's Redpanda Cloud account: +. As a *SQL: Manage* user, grant `SELECT` on the topics they need. The user identifier is the email on the user's Redpanda Cloud account: + [,sql] ---- -GRANT SELECT ON TABLE default_redpanda_catalog=>orders TO "alice@example.com"; ----- -+ -Or grant `SELECT` on multiple tables that match a pattern: -+ -[,sql] ----- -GRANT SELECT ON TABLE default_redpanda_catalog=>orders_* TO "alice@example.com"; +GRANT SELECT ON EXTERNAL SOURCE default_redpanda_catalog => 'orders' TO "alice@example.com"; ---- +For the full access model (privilege levels, wildcards, schema-level permissions, and grant inspection), see xref:sql:manage/manage-access.adoc[Manage access to Redpanda SQL]. + The user can now connect to Redpanda SQL and run `SELECT` against the tables they've been granted. == Next steps -// TODO: Uncomment once DOC-1990 and DOC-2006 merge (target pages are on those branches). +// TODO: Uncomment once DOC-1990, DOC-1999, and DOC-2006 merge (target pages are on those branches). // * xref:sql:query-data/query-streaming-topics.adoc[Query streaming topics]: Map a Redpanda topic to a SQL table and run analytical queries against live streaming data. // * xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics]: Run a single SQL query that spans live records and the Iceberg-translated history of a topic. +// * xref:sql:connect-to-sql/authenticate.adoc[Authenticate to Redpanda SQL]: Connect with an OIDC bearer token, OIDC client credentials, or a SCRAM password for clients that don't support OIDC (for example, Power BI). * xref:reference:sql/index.adoc[Redpanda SQL reference]: Explore the full SQL syntax, data types, functions, and clauses. From 55bf4502bf563c1ecab351fb0ccb205873c1e606 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 20 May 2026 23:06:08 -0700 Subject: [PATCH 14/14] Apply suggestions from SME review --- .../pages/get-started/deploy-sql-cluster.adoc | 32 ++++++++++++++----- .../sql/pages/get-started/sql-quickstart.adoc | 7 +--- 2 files changed, 25 insertions(+), 14 deletions(-) diff --git a/modules/sql/pages/get-started/deploy-sql-cluster.adoc b/modules/sql/pages/get-started/deploy-sql-cluster.adoc index 0c7fedbf8..f49a5faae 100644 --- a/modules/sql/pages/get-started/deploy-sql-cluster.adoc +++ b/modules/sql/pages/get-started/deploy-sql-cluster.adoc @@ -6,17 +6,19 @@ :learning-objective-2: Scale or disable the SQL engine :learning-objective-3: Verify that the SQL engine is running and ready to accept connections -Enable Redpanda SQL on a Bring Your Own Cloud (BYOC) cluster so you can query streaming data in Redpanda topics using standard PostgreSQL syntax. Iceberg-enabled topics can also be queried alongside their Iceberg-translated history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg topics] for that workflow. +Enable Redpanda SQL on a Bring Your Own Cloud (BYOC) cluster so you can query streaming data in Redpanda topics using standard PostgreSQL syntax. For Iceberg-enabled topics, queries can span both the streaming topic and its Iceberg history. See xref:sql:query-data/query-iceberg-topics.adoc[Query Iceberg-enabled topics] for that workflow. -After reading this page, you will be able to: +Use this page to: * [ ] {learning-objective-1} * [ ] {learning-objective-2} * [ ] {learning-objective-3} +NOTE: Redpanda SQL is currently available only on BYOC clusters running on AWS. + == Prerequisites -To enable Redpanda SQL engine, you need: +To enable Redpanda SQL, you need: * A Redpanda Cloud organization on xref:billing:billing.adoc[usage-based billing]. * Admin permissions in your Redpanda Cloud organization. @@ -24,7 +26,7 @@ To enable Redpanda SQL engine, you need: == Enable Redpanda SQL -You can enable Redpanda SQL when you create a new BYOC cluster or on an existing cluster. +You can enable Redpanda SQL on a new or existing BYOC cluster. === On a new cluster @@ -162,18 +164,32 @@ The request returns the ID of a long-running operation. Poll link:/api/doc/cloud After you enable Redpanda SQL, the cluster overview page in the Cloud Console shows the *SQL* tab and the *Details* pane displays the number of SQL nodes deployed with the cluster. // TODO: Confirm UI changes for SQL engine ready indicator -The *SQL* tab appears as soon as you enable SQL, but you can't connect until the engine is fully provisioned. Provisioning can take up to 30 minutes. Wait for the node-ready status indicator on the overview page to show the engine is ready. For the API flow, poll the long-running operation until it returns `STATE_COMPLETED`. +The *SQL* tab appears as soon as you enable SQL, but you can't connect until the engine is fully provisioned. Provisioning can take up to 30 minutes. + +Wait for the node-ready status indicator on the overview page to show the engine is ready. For the API flow, poll the long-running operation until it returns `STATE_COMPLETED`. To verify the SQL engine is running, use the connection details on the *SQL* tab to connect with a PostgreSQL client, such as `psql` (v16 or later required). -The following shows how to connect using a bearer token. Log in to Redpanda Cloud with `rpk`, then retrieve a temporary authentication token for the SQL engine with `rpk cloud auth token` (xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6+] required): +To connect using a bearer token (xref:manage:rpk/rpk-install.adoc[`rpk` v26.1.6+] required): +. Log in to Redpanda Cloud with `rpk cloud login`: ++ [,bash] ---- rpk cloud login +---- +. Retrieve a temporary authentication token for the SQL engine: ++ +[,bash] +---- rpsql_token=$(rpk cloud auth token) +---- +. Connect with `psql` using the bearer token: ++ +[,bash] +---- psql "host= port=5432 dbname=oxla user=ignored password=$rpsql_token options='-c auth_method=bearer' sslmode=require" ---- @@ -194,10 +210,10 @@ SHOW QUERIES; -- List currently running queries [WARNING] ==== -Disabling Redpanda SQL purges the stored catalog state for the SQL engine (catalog metadata, table mappings, and role/grant data) and deletes its data from object storage, including Iceberg-translated data for Iceberg-enabled topics. In-flight queries fail when SQL is disabled. +Disabling Redpanda SQL tears down the SQL compute engine and clears its catalog state (catalog metadata, table mappings, and role/grant data). In-flight queries fail when SQL is disabled. ==== -Redpanda topic data and Schema Registry subjects are not affected. The Redpanda cluster itself continues to run normally; only the SQL engine and its associated state are removed. +Redpanda topic data, Schema Registry subjects, and any Iceberg-committed history for Iceberg-enabled topics are not affected. The Redpanda cluster itself continues to run normally; only the SQL engine and its associated state are removed. Re-enabling SQL on the same cluster provisions a fresh engine: no prior catalog state, table mappings, or grants are restored. You must re-create catalogs, tables, and grants after re-enabling. diff --git a/modules/sql/pages/get-started/sql-quickstart.adoc b/modules/sql/pages/get-started/sql-quickstart.adoc index 3ae6e2edd..37e898565 100644 --- a/modules/sql/pages/get-started/sql-quickstart.adoc +++ b/modules/sql/pages/get-started/sql-quickstart.adoc @@ -28,7 +28,7 @@ After reading this page, you will be able to: Skip this section if you already have a Redpanda topic with a schema registered in Schema Registry that you want to query. ==== -If you don't have a schema-registered topic to query yet, follow these steps to create an `orders` topic with a small set of sample records. Redpanda SQL reads the topic's schema from Schema Registry to map fields to SQL columns, so the topic must have a registered schema before you can query it. +If you don't have a topic with a corresponding schema in Schema Registry yet, follow these steps to create an `orders` topic with a small set of sample records. You also need permissions to create topics, register schemas, and produce records. @@ -175,11 +175,6 @@ LIMIT 20; == Grant access to a non-admin user -[TIP] -==== -Skip this section if you don't need to share SQL access yet. -==== - To share SQL access with a teammate or service account: . In Redpanda Cloud, assign the user a role with the *SQL: Access* permission (see xref:security:authorization/rbac/rbac_dp.adoc[Configure RBAC in the Data Plane]). Redpanda Cloud provisions a corresponding user in the SQL engine.