diff --git a/README.md b/README.md index d62ead4..b8ab936 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,8 @@ $ helm install kube-prometheus-stack kube-prometheus-stack To deploy Gthulhu using Helm charts, follow these steps: ```bash -$ helm install gthulhu gthulhu -f ./gthulhu/values-production.yaml +$ cd gthulhu +$ helm install gthulhu . -f ./values-production.yaml --set mtls.enabled=true --set-file mtls.ca.cert=certs/ca.crt --set-file mtls.dm.cert=certs/dm.crt --set-file mtls.dm.key=certs/dm.key --set-file mtls.manager.cert=certs/manager.crt --set-file mtls.manager.key=certs/manager.key ``` To uninstall Gthulhu, run the following command: diff --git a/gthulhu/.helmignore b/gthulhu/.helmignore index 0e8a0eb..f7a44b0 100644 --- a/gthulhu/.helmignore +++ b/gthulhu/.helmignore @@ -21,3 +21,7 @@ .idea/ *.tmproj .vscode/ + +# Generated mTLS certificates +certs/ +gen-mtls-certs.sh diff --git a/gthulhu/README.md b/gthulhu/README.md index 03703f2..fb4359a 100644 --- a/gthulhu/README.md +++ b/gthulhu/README.md @@ -214,6 +214,150 @@ kubectl logs -l app.kubernetes.io/component=api helm uninstall gthulhu ``` +## mTLS (Mutual TLS) + +Gthulhu supports **mutual TLS** to authenticate and encrypt traffic on two communication paths: + +| Path | Client | Server | Notes | +|------|--------|--------|-------| +| Manager → DM sidecar | Manager (Deployment) | DM sidecar (DaemonSet) | Cross-node; protects scheduling intents | +| Scheduler → DM sidecar | Scheduler (same Pod) | DM sidecar (same Pod) | Loopback; protects the local API call | + +Both paths share a single **private CA** — every certificate is signed by this CA so each peer can verify the other. + +### Quick Start + +```bash +# 1. Generate a private CA + leaf certificates +./gen-mtls-certs.sh certs + +# 2. Install the chart with mTLS enabled +helm install gthulhu ./gthulhu \ + --set mtls.enabled=true \ + --set-file mtls.ca.cert=certs/ca.crt \ + --set-file mtls.dm.cert=certs/dm.crt \ + --set-file mtls.dm.key=certs/dm.key \ + --set-file mtls.manager.cert=certs/manager.crt \ + --set-file mtls.manager.key=certs/manager.key +``` + +### Using Your Own Certificates + +If you already have a PKI or want to bring your own certificates, follow the steps below. + +#### 1. Create a Private CA + +```bash +# EC P-256 key (recommended); RSA-4096 also works +openssl ecparam -name prime256v1 -genkey -noout -out ca.key + +# Self-signed CA certificate (10-year validity) +openssl req -new -x509 -days 3650 \ + -key ca.key -out ca.crt \ + -subj "/CN=Gthulhu-Private-CA" +``` + +#### 2. Generate the DM Sidecar Server Certificate + +The DM sidecar is the TLS **server**. Its certificate needs a `subjectAltName` that covers +`localhost` and `127.0.0.1` (so the in-pod scheduler can connect) plus any DNS names +the Manager uses to reach it (e.g. `*.svc.cluster.local`). + +```bash +openssl ecparam -name prime256v1 -genkey -noout -out dm.key + +openssl req -new -key dm.key -out dm.csr \ + -subj "/CN=gthulhu-decisionmaker" + +openssl x509 -req -days 730 \ + -in dm.csr -CA ca.crt -CAkey ca.key -CAcreateserial \ + -extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:*.svc.cluster.local\nextendedKeyUsage=serverAuth,clientAuth") \ + -out dm.crt +``` + +#### 3. Generate the Manager / Scheduler Client Certificate + +The Manager and the Scheduler both act as mTLS **clients** when talking to the DM sidecar. +They share the same client certificate. + +```bash +openssl ecparam -name prime256v1 -genkey -noout -out manager.key + +openssl req -new -key manager.key -out manager.csr \ + -subj "/CN=gthulhu-manager" + +openssl x509 -req -days 730 \ + -in manager.csr -CA ca.crt -CAkey ca.key -CAcreateserial \ + -extfile <(printf "extendedKeyUsage=clientAuth") \ + -out manager.crt +``` + +#### 4. Supply the Certificates to the Chart + +**Option A — inline via `--set-file`:** + +```bash +helm install gthulhu ./gthulhu \ + --set mtls.enabled=true \ + --set-file mtls.ca.cert=ca.crt \ + --set-file mtls.dm.cert=dm.crt \ + --set-file mtls.dm.key=dm.key \ + --set-file mtls.manager.cert=manager.crt \ + --set-file mtls.manager.key=manager.key +``` + +**Option B — pre-created Kubernetes Secret:** + +```bash +kubectl create secret generic my-gthulhu-mtls \ + --from-file=ca.crt \ + --from-file=dm.crt \ + --from-file=dm.key \ + --from-file=manager.crt \ + --from-file=manager.key + +helm install gthulhu ./gthulhu \ + --set mtls.enabled=true \ + --set mtls.existingSecret=my-gthulhu-mtls +``` + +### Certificate Rotation + +Because the ConfigMap and the scheduler mTLS config Secret are created with +`immutable: true`, you must use `helm upgrade --force` (which deletes and +recreates immutable resources) when rotating certificates: + +```bash +helm upgrade gthulhu ./gthulhu --force \ + --set mtls.enabled=true \ + --set-file mtls.ca.cert=new-ca.crt \ + --set-file mtls.dm.cert=new-dm.crt \ + --set-file mtls.dm.key=new-dm.key \ + --set-file mtls.manager.cert=new-manager.crt \ + --set-file mtls.manager.key=new-manager.key +``` + +### mTLS Configuration Reference + +| Parameter | Description | Default | +|-----------|-------------|---------| +| `mtls.enabled` | Enable mutual TLS | `false` | +| `mtls.existingSecret` | Name of a pre-created Secret containing all PEM files | `""` | +| `mtls.ca.cert` | PEM-encoded CA certificate | `""` | +| `mtls.dm.cert` | PEM-encoded DM sidecar server certificate | `""` | +| `mtls.dm.key` | PEM-encoded DM sidecar server private key | `""` | +| `mtls.manager.cert` | PEM-encoded Manager/Scheduler client certificate | `""` | +| `mtls.manager.key` | PEM-encoded Manager/Scheduler client private key | `""` | + +### Architecture Notes + +- The Manager's **external HTTP API** (web GUI / Ingress) remains **plain HTTP**. + Use a Kubernetes Ingress with TLS termination for external HTTPS. +- When mTLS is enabled, health-check probes on the DM sidecar switch from + `httpGet` to `tcpSocket` because the kubelet cannot present a client certificate. +- The scheduler config is stored in a Kubernetes **Secret** (not a ConfigMap) + when mTLS is enabled, because it contains private-key material inline. + ## Development ### Testing the Chart diff --git a/gthulhu/certs/ca.crt b/gthulhu/certs/ca.crt new file mode 100644 index 0000000..2500dc9 --- /dev/null +++ b/gthulhu/certs/ca.crt @@ -0,0 +1,11 @@ +-----BEGIN CERTIFICATE----- +MIIBjzCCATWgAwIBAgIUSJ2qiDFEvwKXlPAGWOnaD3onTGQwCgYIKoZIzj0EAwIw +HTEbMBkGA1UEAwwSR3RodWxodS1Qcml2YXRlLUNBMB4XDTI2MDIyMzAzMTM0NFoX +DTM2MDIyMTAzMTM0NFowHTEbMBkGA1UEAwwSR3RodWxodS1Qcml2YXRlLUNBMFkw +EwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEsrU+DZyBpiuB5pvSuS/fbOGckLlwTC73 +mkQ7hnWwDnA/q+baN77PVtoVZrGNwxxgD0FAhzdrkRAf2jfEuTiqfqNTMFEwHQYD +VR0OBBYEFIMViDVVNYHUrpQd8MIRbGMk+zjUMB8GA1UdIwQYMBaAFIMViDVVNYHU +rpQd8MIRbGMk+zjUMA8GA1UdEwEB/wQFMAMBAf8wCgYIKoZIzj0EAwIDSAAwRQIg +Ro9LhiUYzT7F5iXEJ0IM64quGDGlSWrbiRBO2odpTq0CIQDMEBxBhsqlx7h4ok6D +vfoIMENsvJMMzAwN8uA6Y61lKA== +-----END CERTIFICATE----- diff --git a/gthulhu/certs/ca.key b/gthulhu/certs/ca.key new file mode 100644 index 0000000..0d9db4d --- /dev/null +++ b/gthulhu/certs/ca.key @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIIzkO4hC30h7tkNNeCt/NUej6NYWubQ1N6D7vef8uX9yoAoGCCqGSM49 +AwEHoUQDQgAEsrU+DZyBpiuB5pvSuS/fbOGckLlwTC73mkQ7hnWwDnA/q+baN77P +VtoVZrGNwxxgD0FAhzdrkRAf2jfEuTiqfg== +-----END EC PRIVATE KEY----- diff --git a/gthulhu/certs/dm.crt b/gthulhu/certs/dm.crt new file mode 100644 index 0000000..e82f503 --- /dev/null +++ b/gthulhu/certs/dm.crt @@ -0,0 +1,16 @@ +-----BEGIN CERTIFICATE----- +MIICdjCCAhygAwIBAgIUCT9p4TDp3lc2Fwp+EfecoNsTVVgwCgYIKoZIzj0EAwIw +HTEbMBkGA1UEAwwSR3RodWxodS1Qcml2YXRlLUNBMB4XDTI2MDIyMzAzMTM0NVoX +DTI4MDIyMzAzMTM0NVowIDEeMBwGA1UEAwwVZ3RodWxodS1kZWNpc2lvbm1ha2Vy +MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEI9rucwma2o8H2eLPg+DLOlMqxEYp +rOWqFw6xdWrNt6FYcRpbvPeO1R75ef86pmQzdIt8N87kvRpk1UMsXW8McqOCATUw +ggExMIHPBgNVHREEgccwgcSCCWxvY2FsaG9zdIcEfwAAAYI9Ki5ndGh1bGh1LWd0 +aHVsaHUtc2NoZWR1bGVyLXNpZGVjYXIuZGVmYXVsdC5zdmMuY2x1c3Rlci5sb2Nh +bIIvKi5ndGh1bGh1LWd0aHVsaHUtc2NoZWR1bGVyLXNpZGVjYXIuZGVmYXVsdC5z +dmOCO2d0aHVsaHUtZ3RodWxodS1zY2hlZHVsZXItc2lkZWNhci5kZWZhdWx0LnN2 +Yy5jbHVzdGVyLmxvY2FshwQKAQAbMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF +BQcDAjAdBgNVHQ4EFgQU+5CDmEm5uoIDQ+cMpwVIaLhVk1kwHwYDVR0jBBgwFoAU +gxWINVU1gdSulB3wwhFsYyT7ONQwCgYIKoZIzj0EAwIDSAAwRQIhALAeHhtuVmV9 +runXu4ssKoxuH5EbeeGaQPgSFn+rDzTbAiByGECW4rDlc7EtHtj3YoFBjDPIJ5RM +YLrD4+pcyXXXpA== +-----END CERTIFICATE----- diff --git a/gthulhu/certs/dm.key b/gthulhu/certs/dm.key new file mode 100644 index 0000000..ac75232 --- /dev/null +++ b/gthulhu/certs/dm.key @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMEPRqDJaZ1/86TCIOWL9TxxIbh/y+LJPi7IUj7nZ1BooAoGCCqGSM49 +AwEHoUQDQgAEI9rucwma2o8H2eLPg+DLOlMqxEYprOWqFw6xdWrNt6FYcRpbvPeO +1R75ef86pmQzdIt8N87kvRpk1UMsXW8Mcg== +-----END EC PRIVATE KEY----- diff --git a/gthulhu/certs/manager.crt b/gthulhu/certs/manager.crt new file mode 100644 index 0000000..296ca2a --- /dev/null +++ b/gthulhu/certs/manager.crt @@ -0,0 +1,11 @@ +-----BEGIN CERTIFICATE----- +MIIBkDCCATagAwIBAgIUCT9p4TDp3lc2Fwp+EfecoNsTVVkwCgYIKoZIzj0EAwIw +HTEbMBkGA1UEAwwSR3RodWxodS1Qcml2YXRlLUNBMB4XDTI2MDIyMzAzMTM0NVoX +DTI4MDIyMzAzMTM0NVowGjEYMBYGA1UEAwwPZ3RodWxodS1tYW5hZ2VyMFkwEwYH +KoZIzj0CAQYIKoZIzj0DAQcDQgAEc6EOY5fjmiO/ffE3yHcQBnDINRAE1GiBk0xc +eRny/6dELBcWO48TbSXw4uu6YD3zczk2gq5jKKSFM8WUUmSxVaNXMFUwEwYDVR0l +BAwwCgYIKwYBBQUHAwIwHQYDVR0OBBYEFHtY+SaE/DxkUtwY2AlWdk/PeoD4MB8G +A1UdIwQYMBaAFIMViDVVNYHUrpQd8MIRbGMk+zjUMAoGCCqGSM49BAMCA0gAMEUC +IHQugTGHFp986lKCEleY0aRgw5xU46eRgLOVhw20GzxDAiEAlk8BmsmPXQTrsIA9 +n9cGKdtJjHC91Lb/RTydrSH73w4= +-----END CERTIFICATE----- diff --git a/gthulhu/certs/manager.key b/gthulhu/certs/manager.key new file mode 100644 index 0000000..4ee6e4b --- /dev/null +++ b/gthulhu/certs/manager.key @@ -0,0 +1,5 @@ +-----BEGIN EC PRIVATE KEY----- +MHcCAQEEIMb/LITD2EAFHDfryINaeuh/cVgLDac36J4weDztgkOyoAoGCCqGSM49 +AwEHoUQDQgAEc6EOY5fjmiO/ffE3yHcQBnDINRAE1GiBk0xceRny/6dELBcWO48T +bSXw4uu6YD3zczk2gq5jKKSFM8WUUmSxVQ== +-----END EC PRIVATE KEY----- diff --git a/gthulhu/gen-mtls-certs.sh b/gthulhu/gen-mtls-certs.sh new file mode 100755 index 0000000..a260754 --- /dev/null +++ b/gthulhu/gen-mtls-certs.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# gen-mtls-certs.sh — Generate a private CA and leaf certificates for Gthulhu mTLS. +# +# Usage: +# ./gen-mtls-certs.sh [OUTPUT_DIR] +# +# Environment variables: +# DM_EXTRA_SANS — Extra SANs for the DM server certificate (comma-separated). +# Example: DM_EXTRA_SANS="IP:10.0.0.5,DNS:my-dm.example.com" +# HELM_RELEASE — Helm release name (default: gthulhu). Used to build the +# headless-service wildcard DNS SAN. +# NAMESPACE — Kubernetes namespace (default: default). +# +# Outputs (all PEM-encoded): +# ca.crt / ca.key — Private CA +# dm.crt / dm.key — Decision Maker (DM sidecar) server certificate +# manager.crt / manager.key — Manager client certificate +# +# The DM server cert includes SANs for: +# - localhost / 127.0.0.1 (scheduler → sidecar, same pod) +# - *.-gthulhu-scheduler-sidecar..svc.cluster.local +# (manager → sidecar, cross-node via headless svc DNS) +# - Any extra SANs from DM_EXTRA_SANS + +set -euo pipefail + +OUT="${1:-certs}" +mkdir -p "$OUT" + +CA_DAYS=3650 # 10 years +LEAF_DAYS=730 # 2 years + +RELEASE="${HELM_RELEASE:-gthulhu}" +NS="${NAMESPACE:-default}" +HEADLESS_SVC="${RELEASE}-gthulhu-scheduler-sidecar" + +# Build DM SAN string +DM_SANS="DNS:localhost,IP:127.0.0.1" +DM_SANS="${DM_SANS},DNS:*.${HEADLESS_SVC}.${NS}.svc.cluster.local" +DM_SANS="${DM_SANS},DNS:*.${HEADLESS_SVC}.${NS}.svc" +DM_SANS="${DM_SANS},DNS:${HEADLESS_SVC}.${NS}.svc.cluster.local" +if [[ -n "${DM_EXTRA_SANS:-}" ]]; then + DM_SANS="${DM_SANS},${DM_EXTRA_SANS}" +fi + +echo "==> DM certificate SANs: ${DM_SANS}" +echo "" + +echo "==> Generating private CA …" +openssl ecparam -name prime256v1 -genkey -noout -out "$OUT/ca.key" +openssl req -new -x509 -days "$CA_DAYS" \ + -key "$OUT/ca.key" \ + -out "$OUT/ca.crt" \ + -subj "/CN=Gthulhu-Private-CA" + +echo "==> Generating DM sidecar server certificate …" +openssl ecparam -name prime256v1 -genkey -noout -out "$OUT/dm.key" +openssl req -new \ + -key "$OUT/dm.key" \ + -out "$OUT/dm.csr" \ + -subj "/CN=gthulhu-decisionmaker" +openssl x509 -req -days "$LEAF_DAYS" \ + -in "$OUT/dm.csr" \ + -CA "$OUT/ca.crt" -CAkey "$OUT/ca.key" -CAcreateserial \ + -extfile <(printf "subjectAltName=${DM_SANS}\nextendedKeyUsage=serverAuth,clientAuth") \ + -out "$OUT/dm.crt" + +echo "==> Generating Manager client certificate …" +openssl ecparam -name prime256v1 -genkey -noout -out "$OUT/manager.key" +openssl req -new \ + -key "$OUT/manager.key" \ + -out "$OUT/manager.csr" \ + -subj "/CN=gthulhu-manager" +openssl x509 -req -days "$LEAF_DAYS" \ + -in "$OUT/manager.csr" \ + -CA "$OUT/ca.crt" -CAkey "$OUT/ca.key" -CAcreateserial \ + -extfile <(printf "extendedKeyUsage=clientAuth") \ + -out "$OUT/manager.crt" + +# Clean up CSRs +rm -f "$OUT"/*.csr "$OUT"/*.srl + +echo "" +echo "✅ Certificates generated in $OUT/" +echo "" +echo "Files:" +ls -1 "$OUT" +echo "" +echo "To install with mTLS enabled:" +echo " helm install gthulhu ./gthulhu \\" +echo " --set mtls.enabled=true \\" +echo " --set-file mtls.ca.cert=$OUT/ca.crt \\" +echo " --set-file mtls.dm.cert=$OUT/dm.crt \\" +echo " --set-file mtls.dm.key=$OUT/dm.key \\" +echo " --set-file mtls.manager.cert=$OUT/manager.crt \\" +echo " --set-file mtls.manager.key=$OUT/manager.key" diff --git a/gthulhu/templates/_helpers.tpl b/gthulhu/templates/_helpers.tpl index b69a89a..bc7a519 100644 --- a/gthulhu/templates/_helpers.tpl +++ b/gthulhu/templates/_helpers.tpl @@ -78,3 +78,14 @@ MongoDB auth secret name (referencing subchart) {{- printf "%s-mongodb-auth" .Release.Name }} {{- end }} {{- end }} + +{{/* +mTLS Secret name — returns existingSecret if set, otherwise the chart-managed name. +*/}} +{{- define "gthulhu.mtlsSecretName" -}} +{{- if .Values.mtls.existingSecret }} +{{- .Values.mtls.existingSecret }} +{{- else }} +{{- printf "%s-mtls-certs" (include "gthulhu.fullname" .) }} +{{- end }} +{{- end }} diff --git a/gthulhu/templates/configmap.yaml b/gthulhu/templates/configmap.yaml index f9659e8..7a7a538 100644 --- a/gthulhu/templates/configmap.yaml +++ b/gthulhu/templates/configmap.yaml @@ -6,6 +6,7 @@ metadata: labels: {{- include "gthulhu.labels" . | nindent 4 }} app.kubernetes.io/component: scheduler +immutable: true data: jwt_public_key.pem: | -----BEGIN PUBLIC KEY----- diff --git a/gthulhu/templates/deployment.yaml b/gthulhu/templates/deployment.yaml index c3ec609..8ee9240 100644 --- a/gthulhu/templates/deployment.yaml +++ b/gthulhu/templates/deployment.yaml @@ -30,6 +30,9 @@ spec: {{- end }} serviceAccountName: {{ include "gthulhu.serviceAccountName" . }} hostPID: {{ .Values.scheduler.hostPID }} + {{- if .Values.scheduler.sidecar.enabled }} + subdomain: {{ include "gthulhu.fullname" . }}-scheduler-sidecar + {{- end }} containers: - name: gthulhu-scheduler image: "{{ .Values.scheduler.image.repository }}:{{ .Values.scheduler.image.tag | default .Chart.AppVersion }}" @@ -65,6 +68,25 @@ spec: value: {{ .Values.scheduler.sidecar.env.loggingLevel | quote }} - name: TZ value: {{ .Values.global.timezone | quote }} + {{- if .Values.mtls.enabled }} + - name: DM_MTLS_ENABLE + value: "true" + - name: DM_MTLS_CERT_PEM + valueFrom: + secretKeyRef: + name: {{ include "gthulhu.mtlsSecretName" . }} + key: dm.crt + - name: DM_MTLS_KEY_PEM + valueFrom: + secretKeyRef: + name: {{ include "gthulhu.mtlsSecretName" . }} + key: dm.key + - name: DM_MTLS_CA_PEM + valueFrom: + secretKeyRef: + name: {{ include "gthulhu.mtlsSecretName" . }} + key: ca.crt + {{- end }} securityContext: {{- toYaml .Values.scheduler.sidecar.securityContext | nindent 12 }} ports: @@ -72,6 +94,22 @@ spec: containerPort: {{ .Values.scheduler.sidecar.port }} protocol: TCP {{- if .Values.scheduler.sidecar.healthCheck.enabled }} + {{- if .Values.mtls.enabled }} + readinessProbe: + tcpSocket: + port: {{ .Values.scheduler.sidecar.port }} + initialDelaySeconds: {{ .Values.scheduler.sidecar.healthCheck.initialDelaySeconds }} + periodSeconds: {{ .Values.scheduler.sidecar.healthCheck.periodSeconds }} + timeoutSeconds: {{ .Values.scheduler.sidecar.healthCheck.timeoutSeconds }} + failureThreshold: {{ .Values.scheduler.sidecar.healthCheck.failureThreshold }} + livenessProbe: + tcpSocket: + port: {{ .Values.scheduler.sidecar.port }} + initialDelaySeconds: {{ add .Values.scheduler.sidecar.healthCheck.initialDelaySeconds 5 }} + periodSeconds: {{ mul .Values.scheduler.sidecar.healthCheck.periodSeconds 2 }} + timeoutSeconds: {{ .Values.scheduler.sidecar.healthCheck.timeoutSeconds }} + failureThreshold: {{ .Values.scheduler.sidecar.healthCheck.failureThreshold }} + {{- else }} readinessProbe: httpGet: path: {{ .Values.scheduler.sidecar.healthCheck.path }} @@ -89,6 +127,7 @@ spec: timeoutSeconds: {{ .Values.scheduler.sidecar.healthCheck.timeoutSeconds }} failureThreshold: {{ .Values.scheduler.sidecar.healthCheck.failureThreshold }} {{- end }} + {{- end }} volumeMounts: - name: proc mountPath: /proc @@ -105,9 +144,25 @@ spec: - name: proc hostPath: path: /proc + {{- if .Values.mtls.enabled }} + - name: config-volume + projected: + sources: + - configMap: + name: {{ include "gthulhu.fullname" . }}-scheduler-config + items: + - key: jwt_public_key.pem + path: jwt_public_key.pem + - secret: + name: {{ include "gthulhu.fullname" . }}-scheduler-mtls-config + items: + - key: config.yaml + path: config.yaml + {{- else }} - name: config-volume configMap: name: {{ include "gthulhu.fullname" . }}-scheduler-config + {{- end }} - name: var-run hostPath: path: /var/run @@ -171,6 +226,29 @@ spec: value: {{ .Values.manager.env.inCluster | quote }} - name: TZ value: {{ .Values.global.timezone | quote }} + {{- if .Values.mtls.enabled }} + - name: MANAGER_MTLS_ENABLE + value: "true" + - name: MANAGER_MTLS_CERT_PEM + valueFrom: + secretKeyRef: + name: {{ include "gthulhu.mtlsSecretName" . }} + key: manager.crt + - name: MANAGER_MTLS_KEY_PEM + valueFrom: + secretKeyRef: + name: {{ include "gthulhu.mtlsSecretName" . }} + key: manager.key + - name: MANAGER_MTLS_CA_PEM + valueFrom: + secretKeyRef: + name: {{ include "gthulhu.mtlsSecretName" . }} + key: ca.crt + {{- if .Values.mtls.serverName }} + - name: MANAGER_MTLS_SERVER_NAME + value: {{ .Values.mtls.serverName | quote }} + {{- end }} + {{- end }} {{- if .Values.mongodb.enabled }} - name: MANAGER_MONGODB_HOST value: {{ include "gthulhu.mongodbHost" . | quote }} diff --git a/gthulhu/templates/mtls-secret.yaml b/gthulhu/templates/mtls-secret.yaml new file mode 100644 index 0000000..9119958 --- /dev/null +++ b/gthulhu/templates/mtls-secret.yaml @@ -0,0 +1,68 @@ +{{/* + mTLS Secret — raw PEM files consumed by Manager and DM sidecar via env vars. + Created only when mtls.enabled=true and no existingSecret is provided. +*/}} +{{- if and .Values.mtls.enabled (not .Values.mtls.existingSecret) }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "gthulhu.fullname" . }}-mtls-certs + labels: + {{- include "gthulhu.labels" . | nindent 4 }} +type: Opaque +stringData: + ca.crt: | + {{- .Values.mtls.ca.cert | nindent 4 }} + dm.crt: | + {{- .Values.mtls.dm.cert | nindent 4 }} + dm.key: | + {{- .Values.mtls.dm.key | nindent 4 }} + manager.crt: | + {{- .Values.mtls.manager.cert | nindent 4 }} + manager.key: | + {{- .Values.mtls.manager.key | nindent 4 }} +{{- end }} +--- +{{/* + Scheduler mTLS config overlay — a Secret that contains the full scheduler + config.yaml with PEM material inlined. Mounted via projected volume so it + overrides the ConfigMap's config.yaml while keeping jwt_public_key.pem from + the ConfigMap. +*/}} +{{- if and .Values.scheduler.enabled .Values.mtls.enabled }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "gthulhu.fullname" . }}-scheduler-mtls-config + labels: + {{- include "gthulhu.labels" . | nindent 4 }} + app.kubernetes.io/component: scheduler +type: Opaque +immutable: true +stringData: + config.yaml: | + # Gthulhu Scheduler Configuration (mTLS enabled) + scheduler: + slice_ns_default: 20000000 + slice_ns_min: 1000000 + mode: gthulhu + kernel_mode: true + max_time_watchdog: true + api: + enabled: true + auth_enabled: true + url: https://localhost:{{ .Values.scheduler.sidecar.port }} + interval: 5 + public_key_path: /etc/gthulhu/jwt_public_key.pem + mtls: + enable: true + cert_pem: | + {{- .Values.mtls.manager.cert | nindent 10 }} + key_pem: | + {{- .Values.mtls.manager.key | nindent 10 }} + ca_pem: | + {{- .Values.mtls.ca.cert | nindent 10 }} + debug: false + early_processing: false + builtin_idle: false +{{- end }} diff --git a/gthulhu/values-production.yaml b/gthulhu/values-production.yaml index 20984f0..045ca40 100644 --- a/gthulhu/values-production.yaml +++ b/gthulhu/values-production.yaml @@ -9,7 +9,7 @@ scheduler: image: repository: ghcr.io/gthulhu/gthulhu - pullPolicy: IfNotPresent + pullPolicy: Always tag: "latest" # Scheduler requires privileged access for BPF operations @@ -50,7 +50,7 @@ scheduler: enabled: true image: - repository: ghcr.io/gthulhu/api + repository: 127.0.0.1:32000/gthulhu-api pullPolicy: Always tag: "latest" @@ -100,7 +100,7 @@ manager: replicaCount: 1 image: - repository: ghcr.io/gthulhu/api + repository: 127.0.0.1:32000/gthulhu-api pullPolicy: Always tag: "latest" @@ -212,6 +212,22 @@ mongodb: runAsGroup: 999 fsGroup: 999 +# mTLS Configuration — enable for production to secure Manager ↔ DM sidecar traffic +mtls: + enabled: true + # Provide your own Secret with keys: ca.crt, dm.crt, dm.key, manager.crt, manager.key + # Generate certs with: ./gen-mtls-certs.sh certs + existingSecret: "" + ca: + cert: "" + dm: + cert: "" + key: "" + manager: + cert: "" + key: "" + serverName: "localhost" + # Global configuration global: imagePullSecrets: [] diff --git a/gthulhu/values.yaml b/gthulhu/values.yaml index 08dfc7d..55a09e3 100644 --- a/gthulhu/values.yaml +++ b/gthulhu/values.yaml @@ -11,9 +11,9 @@ scheduler: replicaCount: 1 image: - repository: 127.0.0.1:32000/gthulhu + repository: ghcr.io/gthulhu/gthulhu pullPolicy: Always - tag: "latest" + tag: "0.6.0" # Scheduler requires privileged access for BPF operations securityContext: @@ -53,9 +53,9 @@ scheduler: enabled: true image: - repository: 127.0.0.1:32000/gthulhu-api + repository: ghcr.io/gthulhu/api pullPolicy: Always - tag: "latest" + tag: "1.3.0" # Sidecar port configuration port: 8080 @@ -103,9 +103,9 @@ manager: replicaCount: 1 image: - repository: 127.0.0.1:32000/gthulhu-api + repository: ghcr.io/gthulhu/api pullPolicy: Always - tag: "latest" + tag: "1.3.0" # Manager port configuration port: 8080 @@ -215,6 +215,33 @@ mongodb: runAsGroup: 999 fsGroup: 999 +# mTLS Configuration +# Mutual TLS for secure communication between components: +# - Manager → DM sidecar (cross-node, Manager is client, DM is server) +# - Scheduler → DM sidecar (same pod via localhost, Scheduler is client, DM is server) +mtls: + enabled: false + # Use an existing Secret instead of the chart-managed one. + # The Secret must contain keys: ca.crt, dm.crt, dm.key, manager.crt, manager.key + existingSecret: "" + # PEM-encoded CA certificate (shared root of trust) + ca: + cert: "" + # DM sidecar server certificate and key (signed by CA) + dm: + cert: "" + key: "" + # Manager / Scheduler client certificate and key (signed by CA) + manager: + cert: "" + key: "" + # TLS ServerName override for Manager → DM connections. + # When set, Go's TLS stack verifies the DM certificate against this name + # instead of the connection target IP. Use "localhost" since the DM cert + # has DNS:localhost in its SAN. This avoids dynamic Pod IP issues. + # Requires api repo support for MTLSConfig.server_name field. + serverName: "localhost" + # Global configuration global: imagePullSecrets: []