Web UI for ksqlDB. Make requests and interact with queries or streams using browser instead of CLI. Written on Python, FastAPI and Jinja2.

Checkout image on Docker Hub: https://hub.docker.com/r/deniskrumko/ksqldb-ui

Features

• Write requests to manipulate streams/queries in UI
• View list of existing queries/streams and detailed info
• View stream/queries topology (how data flows from stream to stream)
• Delete existing queries/streams
• Has translated UI - English (by default), Russian

How it works

You can deploy your ksqlDB server using either Interactive or Headless mode

KsqlDB UI works only for servers in Interactive mode because it allows to use REST API to manipulate ksqlDB server

All statements that exist in ksqlDB are listed in their documentation

Limitations

• You can't use RUN SCRIPT statement in this UI because it can only be executed using file
• Authentication is not yet supported

How to use ksqlDB UI

Note: For production purposes use fixed version from available tags instead of deniskrumko/ksqldb-ui:latest

Using docker

# Download image
docker pull deniskrumko/ksqldb-ui:latest

# Run container
# You need to have config/production.toml file in current directory
docker run \
    -p 8080:8080 \
    -v $(PWD)/config:/config \
    --env APP_CONFIG=/config/production.toml \
    deniskrumko/ksqldb-ui:latest

Using docker-compose.yml

1. Write docker-compose.yml file:

services:
  ksqldb-ui:
    image: deniskrumko/ksqldb-ui:latest
    environment:
      APP_CONFIG: /config.toml
    volumes:
      - ./development.toml:/config.toml
    ports:
      - 8080:8080

2. Run docker-compose up -d

3. Open browser and navigate to http://localhost:8080

Checkout working example of ksqlDB + ksqlDB-UI below.

Using kubernetes manifests

deployment.yml:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ksqldb-ui
  namespace: development
  labels: &labels
    your-labels-here: ksqldb-ui
spec:
  progressDeadlineSeconds: 300
  selector:
    matchLabels: *labels
  replicas: 1
  template:
    metadata:
      labels: *labels
    spec:
      volumes:
        - name: config
          configMap:
            name: ksqldb-ui-configmap
      containers:
        - name: ksqldb-ui
          image: deniskrumko/ksqldb-ui:latest
          volumeMounts:
          - name: config
            mountPath: /config/config.toml
            subPath: config.toml
          ports:
            - containerPort: 8080
          env:
            - name: APP_CONFIG
              value: /config/config.toml
          resources:
            limits:
              cpu: "1"
              memory: 128Mi
            requests:
              cpu: "0.1"
              memory: 64Mi

configmap.yml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ksqldb-ui-configmap
  namespace: development
  labels:
    your-labels-here: ksqldb-ui
data:
  config.toml: |
    [servers.development]
    url = 'http://your-development-ksqldb.com'
    topic_link = 'http://your-development-kafka-ui.com/topics/{}'

    [servers.production]
    url = 'http://your-production-ksqldb.com'
    topic_link = 'http://your-production-kafka-ui.com/topics/{}'

Other manifests (like ingress.yml and so on) you can do on your own 👌

Configuration

Using .toml file and APP_CONFIG env var

Take a look at example config file – config/example.toml

To run ksqlDB UI you need to create own config file and add it using APP_CONFIG env var. See "How to use" section above.

Simplest configuration possible:

[servers.localhost]
url = "http://localhost:8090"

Full config documentation - in future...

Using only environment variables

KsqlDB UI settings works using Dynaconf and that means that all settings can be described/overriden using env vars by following rules:

• Add KSQLDB_UI__ prefix to each var
• Nested params separated using double underscores: __
• Use uppercase for env vars

For example, this config.toml:

[http]
timeout = 60

[servers.localhost]
url = 'http://localhost:8080'

[servers.production]
url = 'http://production:8080'
filters = [['Alice', 'Bob'], ['Red', 'Green', 'Yellow']]

... can be replaces using these env vars:

KSQLDB_UI__HTTP__TIMEOUT=60
KSQLDB_UI__SERVERS__LOCALHOST__URL=http://localhost:8080
KSQLDB_UI__SERVERS__PRODUCTION__URL=http://production:8080
KSQLDB_UI__SERVERS__PRODUCTION__FILTERS="[['Alice', 'Bob'], ['Red', 'Green', 'Yellow']]"

Notes:

• APP_CONFIG env var is not needed when using KSQLDB_UI__ env vars, but you can use both
• KSQLDB_UI__ env vars will always override configuration from APP_CONFIG file
• All available env vars can also be seen on /debug page after opening your ksqlDB UI

Working example

In docker-compose.yml there are three components to work with:

• ksqldb
• ksqldb-ui
• Redpanda (this is just like Apache Kafka but better 😎)
• Redpanda UI

To run this example:

1. Download docker-compose.yml locally

2. Run command:

docker-compose up -d

3. Open ksqldb-ui in browser: http://localhost:8080 to create streams/queries

4. Open Redpanda UI in browser: http://localhost:8090 to create topics

API

NOTE: All API requests require selection of specific server by ?s=<server_code> query parameter.

POST /api/request

Proxy request to ksqlDB server.

Response status code is always HTTP 200 (unless ksqldb-ui errors), because HTTP 200 means that we received response from ksqlDB server. For ksqldb errors check information inside response - in success field or in response field.

curl http://localhost:8080/api/request?s=dev \
    -d "{
        \"query\": \"list streamzzz\"
    }"

Response (error, because query is incorrect):

{
  "success": false,
  "data": {
    "query": "list streamzzz",
    "response": {
      "@type": "statement_error",
      "error_code": 40001,
      "message": "line 1:6: Syntax Error\nSyntax error at or near 'streamzzz' at line 1:6",
      "statementText": "list streamzzz;",
      "entities": []
    }
  }
}

POST /api/process_file

Upload file with SQL statements and get response.

Response status code is always HTTP 200 (unless ksqldb-ui errors), because HTTP 200 means that we received response from ksqlDB server. For ksqldb errors check information inside response - in success field or in response field.

Request with request.sql file:

curl -F "file=@./request.sql" http://localhost:8080/api/process_file?s=dev

File content:

list streams;

Response:

{
  "success": true,
  "data": {
    "query": "list streams;",
    "response": [
      {
        "@type": "streams",
        "statementText": "list streams;",
        "streams": [
          {
            "type": "STREAM",
            "name": "MY_COOL_STREAM",
            "topic": "my-cool-stream",
            "keyFormat": "JSON_SR",
            "valueFormat": "JSON_SR",
            "isWindowed": false
          }
        ],
        "warnings": []
      }
    ]
  }
}

Credits

• Powered by Python 3.12, FastAPI and Jinja2
• UI using Bootstrap 5.3
• SQL editor using Ace
• Icons from Google fonts
• Markdown tables from tablesgenerator.com