[doc] Issue in "Installing cluster API provider dependencies"

[frasdav]

Namespace here should be rancher-turtles-system and not cattle-turtles-system?

With cattle-turtles-system:

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: rancher-turtles
  namespace: kube-system
spec:
  chart: oci://registry.suse.com/edge/charts/rancher-turtles-providers
  version: 305.0.4+up0.25.1
  createNamespace: true
  targetNamespace: cattle-turtles-system

fraser@edge01:~> k get pods -A
NAMESPACE                   NAME                                                    READY   STATUS      RESTARTS        AGE
cattle-fleet-local-system   fleet-agent-8c48b9c56-t9wd6                             1/1     Running     2 (8m48s ago)   69m
cattle-fleet-system         fleet-controller-79cfc97857-gs66b                       3/3     Running     9 (8m48s ago)   70m
cattle-fleet-system         gitjob-657d9fbc59-jm8ds                                 1/1     Running     5 (8m48s ago)   70m
cattle-fleet-system         helmops-7774f79665-4l4n7                                1/1     Running     2 (8m48s ago)   70m
cattle-system               helm-operation-2vnhw                                    1/2     Error       0               49s
cattle-system               helm-operation-5t9lw                                    1/2     Error       0               105s
cattle-system               helm-operation-99ffm                                    1/2     Error       0               26s
cattle-system               helm-operation-d8rvj                                    1/2     Error       0               61s
cattle-system               helm-operation-fz2zn                                    0/2     Completed   0               47m
cattle-system               helm-operation-q9sxt                                    1/2     Error       0               38s
cattle-system               helm-operation-qnp27                                    1/2     Error       0               14s
cattle-system               helm-operation-x8fvg                                    1/2     Error       0               73s
cattle-system               rancher-6d89685897-84f6t                                1/1     Running     2 (8m48s ago)   73m
cattle-system               rancher-webhook-b476c666d-m9z7r                         1/1     Running     2 (8m48s ago)   69m
cattle-system               system-upgrade-controller-65d9b4b8b-sfdrd               1/1     Running     2 (8m48s ago)   69m
cattle-turtles-system       rancher-clusterctl-configmap-cleanup-xn5xg              0/1     Completed   0               7s
cert-manager                cert-manager-8455fdbf67-4rxl7                           1/1     Running     2 (8m48s ago)   74m
cert-manager                cert-manager-cainjector-77dbd66479-szdgg                1/1     Running     2 (8m48s ago)   74m
cert-manager                cert-manager-webhook-78dff5f574-f4gjp                   1/1     Running     2 (8m48s ago)   74m
fleet-addon-system          caapf-controller-manager-67f886f4fc-rp87g               2/2     Running     0               116s
kube-system                 etcd-edge01                                             1/1     Running     2               79m
kube-system                 helm-install-metal3-hpqqb                               0/1     Completed   0               47m
kube-system                 helm-install-metallb-btkgh                              0/1     Completed   0               67m
kube-system                 helm-install-rancher-turtles-gq62b                      0/1     Completed   0               2m7s
kube-system                 helm-install-rke2-canal-h2bv2                           0/1     Completed   0               79m
kube-system                 helm-install-rke2-coredns-2szzj                         0/1     Completed   0               79m
kube-system                 helm-install-rke2-ingress-nginx-7pnst                   0/1     Completed   0               79m
kube-system                 helm-install-rke2-metrics-server-vwnjh                  0/1     Completed   0               79m
kube-system                 helm-install-rke2-runtimeclasses-dr56c                  0/1     Completed   0               79m
kube-system                 helm-install-rke2-snapshot-controller-64s5t             0/1     Completed   1               79m
kube-system                 helm-install-rke2-snapshot-controller-crd-7tw9q         0/1     Completed   0               79m
kube-system                 kube-apiserver-edge01                                   1/1     Running     2               79m
kube-system                 kube-controller-manager-edge01                          1/1     Running     4 (8m45s ago)   79m
kube-system                 kube-proxy-edge01                                       1/1     Running     0               8m26s
kube-system                 kube-scheduler-edge01                                   1/1     Running     2 (8m48s ago)   79m
kube-system                 rke2-canal-g2nzg                                        2/2     Running     4 (8m48s ago)   79m
kube-system                 rke2-coredns-rke2-coredns-784bcb7f4d-69n5s              1/1     Running     2 (8m48s ago)   79m
kube-system                 rke2-coredns-rke2-coredns-autoscaler-84ff4dbbb4-87lng   1/1     Running     2 (8m48s ago)   79m
kube-system                 rke2-ingress-nginx-controller-nlm9w                     1/1     Running     2 (8m48s ago)   78m
kube-system                 rke2-metrics-server-7b59bd8854-4qqk2                    1/1     Running     2 (8m48s ago)   78m
kube-system                 rke2-snapshot-controller-9fccd4467-sjvlf                1/1     Running     2 (8m48s ago)   78m
local-path-storage          local-path-provisioner-6646fbc5d8-lkgz7                 1/1     Running     2 (8m48s ago)   70m
metal3-system               baremetal-operator-controller-manager-bf4cc774d-hq5ch   1/1     Running     1 (8m48s ago)   47m
metal3-system               metal3-metal3-ironic-689747bcb9-zw6lb                   3/3     Running     3 (8m48s ago)   47m
metal3-system               metal3-metal3-mariadb-575dd5c588-sbwx5                  1/1     Running     1 (8m48s ago)   47m
metallb-system              metallb-controller-6f48df7bf9-8q7fw                     1/1     Running     2 (8m48s ago)   67m
metallb-system              metallb-speaker-9q9rt                                   1/1     Running     2 (8m48s ago)   67m

helm-operation-n being created and error on repeat.

fraser@edge01:~> k logs helm-operation-5t9lw -n cattle-system
Defaulted container "helm" out of: helm, proxy, init-kubeconfig-volume (init)
helm upgrade --history-max=5 --install=true --labels=catalog.cattle.io/cluster-repo-name=rancher-charts --namespace=cattle-turtles-system --reset-values=true --take-ownership=true --timeout=5m0s --values=/home/shell/helm/values-rancher-turtles-108.0.2-up0.25.2.yaml --version=108.0.2+up0.25.2 --wait=true rancher-turtles /home/shell/helm/rancher-turtles-108.0.2-up0.25.2.tgz
Error: UPGRADE FAILED: failed to create resource: configmaps "core-cluster-api-v1.10.6" is forbidden: unable to create new content in namespace cattle-capi-system because it is being terminated

With rancher-turtles-system:

---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: rancher-turtles
  namespace: kube-system
spec:
  chart: oci://registry.suse.com/edge/charts/rancher-turtles-providers
  version: 305.0.4+up0.25.1
  createNamespace: true
  targetNamespace: rancher-turtles-system

fraser@edge01:~> k get pods -A
NAMESPACE                   NAME                                                    READY   STATUS      RESTARTS        AGE
capm3-system                capm3-controller-manager-6764d4d8cd-769h5               1/1     Running     0               2m41s
cattle-capi-system          capi-controller-manager-84c7c9989b-4wmjm                1/1     Running     2 (5m25s ago)   54m
cattle-fleet-local-system   fleet-agent-8c48b9c56-t9wd6                             1/1     Running     2 (5m25s ago)   55m
cattle-fleet-system         fleet-controller-686c699c75-pdnck                       3/3     Running     0               2m
cattle-fleet-system         gitjob-fc6c6b677-ng6vr                                  1/1     Running     0               2m
cattle-fleet-system         helmops-86b4f9f9f9-cd68k                                1/1     Running     0               2m
cattle-system               helm-operation-7rjmb                                    0/2     Completed   0               55m
cattle-system               helm-operation-fz2zn                                    0/2     Completed   0               33m
cattle-system               helm-operation-g8zlv                                    0/2     Completed   0               54m
cattle-system               helm-operation-hnnf6                                    0/2     Completed   0               56m
cattle-system               helm-operation-lfxwc                                    0/2     Completed   0               55m
cattle-system               helm-operation-wwbzx                                    0/2     Completed   0               54m
cattle-system               helm-operation-wwxml                                    0/2     Completed   0               55m
cattle-system               helm-operation-z8gj6                                    0/2     Completed   0               54m
cattle-system               helm-operation-z97kj                                    0/2     Completed   0               2m9s
cattle-system               rancher-6d89685897-84f6t                                1/1     Running     3 (4m26s ago)   59m
cattle-system               rancher-webhook-b476c666d-m9z7r                         1/1     Running     2 (5m25s ago)   55m
cattle-system               system-upgrade-controller-65d9b4b8b-sfdrd               1/1     Running     2 (5m25s ago)   55m
cattle-turtles-system       rancher-turtles-controller-manager-788c5d7fd9-qfqm7     1/1     Running     2 (5m25s ago)   54m
cert-manager                cert-manager-8455fdbf67-4rxl7                           1/1     Running     2 (5m25s ago)   60m
cert-manager                cert-manager-cainjector-77dbd66479-szdgg                1/1     Running     2 (5m25s ago)   60m
cert-manager                cert-manager-webhook-78dff5f574-f4gjp                   1/1     Running     2 (5m25s ago)   60m
fleet-addon-system          caapf-controller-manager-85d77668c7-4hnn5               2/2     Running     0               2m38s
kube-system                 etcd-edge01                                             1/1     Running     2               65m
kube-system                 helm-install-metal3-hpqqb                               0/1     Completed   0               33m
kube-system                 helm-install-metallb-btkgh                              0/1     Completed   0               53m
kube-system                 helm-install-rancher-turtles-n9hzz                      0/1     Completed   0               3m15s
kube-system                 helm-install-rke2-canal-h2bv2                           0/1     Completed   0               65m
kube-system                 helm-install-rke2-coredns-2szzj                         0/1     Completed   0               65m
kube-system                 helm-install-rke2-ingress-nginx-7pnst                   0/1     Completed   0               65m
kube-system                 helm-install-rke2-metrics-server-vwnjh                  0/1     Completed   0               65m
kube-system                 helm-install-rke2-runtimeclasses-dr56c                  0/1     Completed   0               65m
kube-system                 helm-install-rke2-snapshot-controller-64s5t             0/1     Completed   1               65m
kube-system                 helm-install-rke2-snapshot-controller-crd-7tw9q         0/1     Completed   0               65m
kube-system                 kube-apiserver-edge01                                   1/1     Running     4               65m
kube-system                 kube-controller-manager-edge01                          1/1     Running     5 (4m45s ago)   65m
kube-system                 kube-proxy-edge01                                       1/1     Running     0               5m4s
kube-system                 kube-scheduler-edge01                                   1/1     Running     2 (5m25s ago)   65m
kube-system                 rke2-canal-g2nzg                                        2/2     Running     5 (4m44s ago)   65m
kube-system                 rke2-coredns-rke2-coredns-784bcb7f4d-69n5s              1/1     Running     2 (5m25s ago)   65m
kube-system                 rke2-coredns-rke2-coredns-autoscaler-84ff4dbbb4-87lng   1/1     Running     2 (5m25s ago)   65m
kube-system                 rke2-ingress-nginx-controller-nlm9w                     1/1     Running     2 (5m25s ago)   64m
kube-system                 rke2-metrics-server-7b59bd8854-4qqk2                    1/1     Running     2 (5m25s ago)   64m
kube-system                 rke2-snapshot-controller-9fccd4467-sjvlf                1/1     Running     2 (5m25s ago)   64m
local-path-storage          local-path-provisioner-6646fbc5d8-lkgz7                 1/1     Running     2 (5m25s ago)   56m
metal3-ipam-system          ipam-controller-manager-994bc47f5-l6cw4                 1/1     Running     0               2m36s
metal3-system               baremetal-operator-controller-manager-bf4cc774d-hq5ch   1/1     Running     1 (5m25s ago)   33m
metal3-system               metal3-metal3-ironic-689747bcb9-zw6lb                   3/3     Running     3 (5m25s ago)   33m
metal3-system               metal3-metal3-mariadb-575dd5c588-sbwx5                  1/1     Running     1 (5m25s ago)   33m
metallb-system              metallb-controller-6f48df7bf9-8q7fw                     1/1     Running     2 (5m25s ago)   53m
metallb-system              metallb-speaker-9q9rt                                   1/1     Running     2 (5m25s ago)   53m
rke2-bootstrap-system       rke2-bootstrap-controller-manager-54d7c4566b-n26d9      1/1     Running     0               2m42s
rke2-control-plane-system   rke2-control-plane-controller-manager-6965b4b74-xs65b   1/1     Running     0               2m37s

[ranjinimn]

@frasdav Thank you for reporting this issue. There are 10 instances of cattle-turtles-system in the 3.5 Edge documentation.
Should I make changes in all the sections? Please confirm. Thanks

[hardys]

@frasdav Thank you for reporting this issue. There are 10 instances of cattle-turtles-system in the 3.5 Edge documentation. Should I make changes in all the sections? Please confirm. Thanks

Thanks for the report @frasdav - for Rancher 2.13 the namespace changed, because the rancher-turtles deployment is now integrated into Rancher, which means the cattle-turtles-system namespace is correct for Edge 3.5 (Rancher 2.13) onwards.

As you mention in previous releases it was rancher-turtles-system but I don't think it's correct to use that from 3.5 onwards, I wonder if we can instead figure out why the unable to create new content in namespace cattle-capi-system because it is being terminated is happening?

My guess is the turtles installation failed and is being re-installed, which may be a temporary problem - can you confirm if the issue persists if you wait a while (IME it can take several minutes for the deployments integrated into Rancher to settle, and the turtles/CAPI dependencies become active towards the end of that process)

You can see some more details on this topic in the migration documentation and also see #996

[frasdav]

Hi @hardys. I'm not convinced it's a temporary problem I was having. When I attempted to install into cattle-turtles-system, the chart ended up in this state:

fraser@edge01:~> k logs helm-operation-5t9lw -n cattle-system
Defaulted container "helm" out of: helm, proxy, init-kubeconfig-volume (init)
helm upgrade --history-max=5 --install=true --labels=catalog.cattle.io/cluster-repo-name=rancher-charts --namespace=cattle-turtles-system --reset-values=true --take-ownership=true --timeout=5m0s --values=/home/shell/helm/values-rancher-turtles-108.0.2-up0.25.2.yaml --version=108.0.2+up0.25.2 --wait=true rancher-turtles /home/shell/helm/rancher-turtles-108.0.2-up0.25.2.tgz
Error: UPGRADE FAILED: failed to create resource: configmaps "core-cluster-api-v1.10.6" is forbidden: unable to create new content in namespace cattle-capi-system because it is being terminated

I confirmed that cattle-capi-system was in deleting state. I'll rebuild and re-test to be sure.