diff --git a/scenarios/3-nodes-gitops/README.md b/scenarios/3-nodes-gitops/README.md index 4bbdb331..6a38e1e4 100644 --- a/scenarios/3-nodes-gitops/README.md +++ b/scenarios/3-nodes-gitops/README.md @@ -1,47 +1,64 @@ # 3-Nodes GitOps Scenario -## Overview +Deploys OpenStack on Single Node OpenShift (SNO) with one EDPM compute node using GitOps workflow. This scenario demonstrates true GitOps deployment where git commits drive cluster state changes. -A minimal OpenStack deployment scenario with 3 nodes: 1 controller, 1 -OpenShift master, and 1 compute node. Sets up OpenShift cluster with GitOps -operator for future RHOSO (Red Hat OpenStack Services on OpenShift) deployment. +## GitOps Workflow -## Architecture +This scenario implements incremental GitOps deployment using a local git repository on the controller node: -- **Controller**: DNS, load balancing, and orchestration -- **OpenShift Master**: Single-node cluster running OpenStack control plane -- **Compute Node**: EDPM compute node for workloads -- **GitOps Operator**: Ready for OpenStack service deployment - -## Networks - -- **machine-net**: 192.168.32.0/24 (OpenShift cluster) -- **ctlplane-net**: 192.168.122.0/24 (Control plane) -- **internal-api-net**: 172.17.0.0/24 (OpenStack internal) -- **storage-net**: 172.18.0.0/24 (Storage backend) -- **tenant-net**: 172.19.0.0/24 (Tenant traffic) -- **octavia-net**: 172.23.0.0/24 (Load balancing) +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Controller Node: ~/git/openstack-deployment │ +│ • Git repository with manifests │ +│ • git-daemon serving on port 9418 │ +│ • Source manifests in ~/gitops-manifests/ │ +└────────────────────────┬────────────────────────────────────────┘ + │ git:// protocol + ↓ +┌─────────────────────────────────────────────────────────────────┐ +│ ArgoCD (on OpenShift) │ +│ • Polls git repo every 3 minutes │ +│ • Detects changes and syncs automatically │ +│ • Applies manifests to cluster │ +└────────────────────────┬────────────────────────────────────────┘ + │ kubectl apply + ↓ +┌─────────────────────────────────────────────────────────────────┐ +│ OpenShift Cluster │ +│ • OpenStack operators, networking, control plane, data plane │ +└─────────────────────────────────────────────────────────────────┘ +``` -**Configuration:** Fixed MAC addresses assigned to all interfaces for -consistent networking. +## Deployment Flow -## Node Configuration +The automation deploys OpenStack incrementally through git commits: -**Controller:** +1. **Apply ArgoCD Applications** - Create all Application CRs (watch empty git repo) +2. **Commit operators** → ArgoCD deploys OpenStack operator +3. **Commit secrets** → ArgoCD deploys secrets +4. **Commit network** → ArgoCD deploys networking +5. **Commit controlplane** → ArgoCD deploys control plane services +6. **Commit dataplane** → ArgoCD deploys compute node -- **Machine Net**: 192.168.32.3 -- **MAC**: fa:16:9e:81:f6:05 +Each git commit triggers ArgoCD to detect and deploy the next component automatically. -**Master0:** +## ArgoCD Applications -- **Machine Net**: 192.168.34.10 -- **Control Plane**: 192.168.122.10 -- **MACs**: fa:16:9e:81:f6:10 (machine), fa:16:9e:81:f6:11 (ctlplane) +Five applications deployed in sync-wave order: -**Compute0:** +``` +┌─────────────────────────────────────────────────────────────┐ +│ ArgoCD Applications │ +├─────────────────────────────────────────────────────────────┤ +│ Wave 10: openstack-operators → manifests/operators/ │ +│ Wave 15: openstack-secrets → manifests/secrets/ │ +│ Wave 20: openstack-network → manifests/network/ │ +│ Wave 30: openstack-controlplane → manifests/controlplane/ │ +│ Wave 40: openstack-dataplane → manifests/dataplane/ │ +└─────────────────────────────────────────────────────────────┘ +``` -- **Control Plane**: 192.168.122.100 -- **MAC**: fa:16:9e:81:f6:20 (ctlplane) +Each application references kustomize manifests that combine upstream components from `openstack-k8s-operators/gitops` with SNO-specific patches. ## Usage @@ -50,31 +67,31 @@ consistent networking. ansible-playbook -i inventory.yml bootstrap.yml \ -e @scenarios/3-nodes-gitops/bootstrap_vars.yml \ -e @~/cloud-secrets.yaml - -# Deploy using snapset images -ansible-playbook -i inventory.yml bootstrap.yml \ - -e @scenarios/3-nodes-gitops/bootstrap_vars.yml \ - -e @~/cloud-secrets.yaml \ - -e hotstack_revive_snapshot=true ``` -> **NOTE**: Snapset deployment requires snapset images to be available in your -> OpenStack cloud. See [Hotstack SnapSet documentation](../../docs/hotstack_snapset.md) -> for details. +## Testing GitOps -## Deployment Process +To manually test GitOps reconciliation: + +```bash +# On controller node +cd ~/git/openstack-deployment +# Edit a manifest +vi manifests/operators/openstack-cr.yaml +git add manifests/ +git commit -m "Update OpenStack configuration" +# ArgoCD detects and syncs automatically (within 3 minutes) +``` -1. **Infrastructure**: Heat template deploys infrastructure -2. **OpenShift**: Single-node cluster installation -3. **GitOps Operator**: Install and configure GitOps operator +## Architecture Details -## Requirements +- **Controller**: DNS, load balancing, git-daemon +- **OpenShift Master**: Single-node cluster (SNO) +- **Compute Node**: EDPM compute node +- **Storage**: TopoLVM for dynamic local storage +- **Networking**: MetalLB, NMState, fixed MAC addresses -- **Instances**: 3 total (1 controller, 1 master, 1 compute) -- **Flavors**: hotstack.small, hotstack.large, hotstack.xxlarge -- **Images**: hotstack-controller, ipxe-boot-usb -- **Features**: Multi-network setup, TopoLVM storage, fixed MAC addresses +## Upstream Components -This scenario provides the foundation for GitOps-based RHOSO deployments. -OpenStack service deployment would be handled separately via GitOps -configurations. +Kustomize manifests reference components from: +- https://github.com/openstack-k8s-operators/gitops diff --git a/scenarios/3-nodes-gitops/argocd-apps/openstack-controlplane.yaml b/scenarios/3-nodes-gitops/argocd-apps/openstack-controlplane.yaml new file mode 100644 index 00000000..cf4c2657 --- /dev/null +++ b/scenarios/3-nodes-gitops/argocd-apps/openstack-controlplane.yaml @@ -0,0 +1,20 @@ +--- +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: openstack-controlplane + namespace: openshift-gitops + annotations: + argocd.argoproj.io/sync-wave: "30" +spec: + destination: + server: https://kubernetes.default.svc + project: default + source: + repoURL: git://controller-0.openstack.lab/openstack-deployment + targetRevision: main + path: manifests/controlplane + syncPolicy: + automated: + prune: true + selfHeal: true diff --git a/scenarios/3-nodes-gitops/argocd-apps/openstack-dataplane.yaml b/scenarios/3-nodes-gitops/argocd-apps/openstack-dataplane.yaml new file mode 100644 index 00000000..38457274 --- /dev/null +++ b/scenarios/3-nodes-gitops/argocd-apps/openstack-dataplane.yaml @@ -0,0 +1,20 @@ +--- +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: openstack-dataplane + namespace: openshift-gitops + annotations: + argocd.argoproj.io/sync-wave: "40" +spec: + destination: + server: https://kubernetes.default.svc + project: default + source: + repoURL: git://controller-0.openstack.lab/openstack-deployment + targetRevision: main + path: manifests/dataplane + syncPolicy: + automated: + prune: true + selfHeal: true diff --git a/scenarios/3-nodes-gitops/argocd-apps/openstack-network.yaml b/scenarios/3-nodes-gitops/argocd-apps/openstack-network.yaml new file mode 100644 index 00000000..3642f088 --- /dev/null +++ b/scenarios/3-nodes-gitops/argocd-apps/openstack-network.yaml @@ -0,0 +1,20 @@ +--- +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: openstack-network + namespace: openshift-gitops + annotations: + argocd.argoproj.io/sync-wave: "20" +spec: + destination: + server: https://kubernetes.default.svc + project: default + source: + repoURL: git://controller-0.openstack.lab/openstack-deployment + targetRevision: main + path: manifests/network + syncPolicy: + automated: + prune: true + selfHeal: true diff --git a/scenarios/3-nodes-gitops/argocd-apps/openstack-operators.yaml b/scenarios/3-nodes-gitops/argocd-apps/openstack-operators.yaml new file mode 100644 index 00000000..e595d25f --- /dev/null +++ b/scenarios/3-nodes-gitops/argocd-apps/openstack-operators.yaml @@ -0,0 +1,22 @@ +--- +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: openstack-operators + namespace: openshift-gitops + annotations: + argocd.argoproj.io/sync-wave: "10" +spec: + destination: + server: https://kubernetes.default.svc + project: default + source: + repoURL: git://controller-0.openstack.lab/openstack-deployment + targetRevision: main + path: manifests/operators + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - CreateNamespace=true diff --git a/scenarios/3-nodes-gitops/argocd-apps/openstack-secrets.yaml b/scenarios/3-nodes-gitops/argocd-apps/openstack-secrets.yaml new file mode 100644 index 00000000..fe00e0e3 --- /dev/null +++ b/scenarios/3-nodes-gitops/argocd-apps/openstack-secrets.yaml @@ -0,0 +1,20 @@ +--- +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: openstack-secrets + namespace: openshift-gitops + annotations: + argocd.argoproj.io/sync-wave: "15" +spec: + destination: + server: https://kubernetes.default.svc + project: default + source: + repoURL: git://controller-0.openstack.lab/openstack-deployment + targetRevision: main + path: manifests/secrets + syncPolicy: + automated: + prune: true + selfHeal: true diff --git a/scenarios/3-nodes-gitops/automation-vars.yml b/scenarios/3-nodes-gitops/automation-vars.yml index 484a3438..d0cbcd7c 100644 --- a/scenarios/3-nodes-gitops/automation-vars.yml +++ b/scenarios/3-nodes-gitops/automation-vars.yml @@ -23,6 +23,17 @@ stages: "common/stages/topolvm-stages.yaml.j2") }} + - name: OpenStack Dependencies + documentation: | + Install prerequisite operators required for OpenStack deployment including + MetalLB (load balancer), NMState (network configuration), and Cert-Manager + (certificate management). + stages: >- + {{ + lookup("ansible.builtin.template", + "common/stages/deps-stages.yaml.j2") + }} + - name: GitOps documentation: | Install and configure OpenShift GitOps via kustomize from the rhoso-gitops @@ -33,3 +44,175 @@ stages: lookup("ansible.builtin.template", "common/stages/gitops-stages.yaml.j2") }} + + - name: Sync gitops-manifests to controller + documentation: | + Copy the latest gitops-manifests from the scenario directory to the controller. + This ensures the controller has the most recent manifests for the GitOps workflow. + sync_files: + src: "gitops-manifests/" + dest: "~/gitops-manifests" + + - name: Apply ArgoCD Applications + documentation: | + Apply all ArgoCD Application CRs. These will monitor the git repository + for manifests to deploy. + manifest: argocd-apps/openstack-operators.yaml + + - name: Apply ArgoCD Application - Secrets + documentation: | + Apply ArgoCD Application for secrets. + manifest: argocd-apps/openstack-secrets.yaml + + - name: Apply ArgoCD Application - Network + documentation: | + Apply ArgoCD Application for network. + manifest: argocd-apps/openstack-network.yaml + + - name: Apply ArgoCD Application - ControlPlane + documentation: | + Apply ArgoCD Application for controlplane. + manifest: argocd-apps/openstack-controlplane.yaml + + - name: Apply ArgoCD Application - DataPlane + documentation: | + Apply ArgoCD Application for dataplane. + manifest: argocd-apps/openstack-dataplane.yaml + + - name: Wait for ArgoCD Applications to be initialized + documentation: | + Wait for all ArgoCD Application CRs to have a health status field. + This ensures they are initialized before we start committing manifests. + wait_conditions: + - >- + oc wait -n openshift-gitops application openstack-operators + --for jsonpath='{.status.health.status}' --timeout=60s + - >- + oc wait -n openshift-gitops application openstack-secrets + --for jsonpath='{.status.health.status}' --timeout=60s + - >- + oc wait -n openshift-gitops application openstack-network + --for jsonpath='{.status.health.status}' --timeout=60s + - >- + oc wait -n openshift-gitops application openstack-controlplane + --for jsonpath='{.status.health.status}' --timeout=60s + - >- + oc wait -n openshift-gitops application openstack-dataplane + --for jsonpath='{.status.health.status}' --timeout=60s + + - name: Deploy OpenStack Operators + documentation: | + Commit operators manifests to git and wait for ArgoCD to deploy. + shell: | + set -ex + pushd ~/git/openstack-deployment + mkdir -p manifests + cp -rv ~/gitops-manifests/operators manifests/ + git add manifests/operators/ + git diff --cached --quiet || git commit -m "Add OpenStack operators" + popd + wait_conditions: + - >- + oc wait -n openshift-gitops application openstack-operators + --for jsonpath='{.status.sync.status}'=Synced --timeout=600s + - >- + oc wait -n openshift-gitops application openstack-operators + --for jsonpath='{.status.health.status}'=Healthy --timeout=600s + - >- + oc wait -n openstack-operators openstack openstack + --for condition=Ready --timeout=300s + + - name: Deploy OpenStack Secrets + documentation: | + Create secrets dynamically using SSH keys from controller, + commit to git, and wait for ArgoCD to sync. + shell: | + set -ex + pushd ~/git/openstack-deployment + mkdir -p manifests/secrets + + # Create dataplane SSH secret from controller keys + oc create -n openstack secret generic dataplane-ansible-ssh-private-key-secret \ + --dry-run=client \ + --from-file=ssh-privatekey=/home/zuul/.ssh/id_rsa \ + --from-file=ssh-publickey=/home/zuul/.ssh/id_rsa.pub \ + --type=Opaque -o yaml > manifests/secrets/dataplane-ssh-secret.yaml + + # Copy other secret templates + cp -v ~/gitops-manifests/secrets/kustomization.yaml manifests/secrets/ + cp -v ~/gitops-manifests/secrets/libvirt-secret.yaml manifests/secrets/ + cp -v ~/gitops-manifests/secrets/osp-secret.yaml manifests/secrets/ + + git add manifests/secrets/ + git diff --cached --quiet || git commit -m "Add OpenStack secrets" + popd + wait_conditions: + - >- + oc wait -n openshift-gitops application openstack-secrets + --for jsonpath='{.status.sync.status}'=Synced --timeout=300s + + - name: Deploy OpenStack Network + documentation: | + Commit network manifests to git and wait for ArgoCD to deploy. + shell: | + set -ex + pushd ~/git/openstack-deployment + mkdir -p manifests + cp -rv ~/gitops-manifests/network manifests/ + git add manifests/network/ + git diff --cached --quiet || git commit -m "Add OpenStack network configuration" + popd + wait_conditions: + - >- + oc wait -n openshift-gitops application openstack-network + --for jsonpath='{.status.sync.status}'=Synced --timeout=600s + - >- + oc wait -n openshift-gitops application openstack-network + --for jsonpath='{.status.health.status}'=Healthy --timeout=600s + - >- + oc wait -n openstack nncp --all + --for condition=Available --timeout=300s + + - name: Deploy OpenStack ControlPlane + documentation: | + Commit controlplane manifests to git and wait for ArgoCD to deploy. + shell: | + set -ex + pushd ~/git/openstack-deployment + mkdir -p manifests + cp -rv ~/gitops-manifests/controlplane manifests/ + git add manifests/controlplane/ + git diff --cached --quiet || git commit -m "Add OpenStack control plane" + popd + wait_conditions: + - >- + oc wait -n openshift-gitops application openstack-controlplane + --for jsonpath='{.status.sync.status}'=Synced --timeout=1200s + - >- + oc wait -n openshift-gitops application openstack-controlplane + --for jsonpath='{.status.health.status}'=Healthy --timeout=1200s + - >- + oc wait -n openstack openstackcontrolplane openstack-control-plane + --for condition=Ready --timeout=1800s + + - name: Deploy OpenStack DataPlane + documentation: | + Commit dataplane manifests to git and wait for ArgoCD to deploy. + shell: | + set -ex + pushd ~/git/openstack-deployment + mkdir -p manifests + cp -rv ~/gitops-manifests/dataplane manifests/ + git add manifests/dataplane/ + git diff --cached --quiet || git commit -m "Add OpenStack data plane" + popd + wait_conditions: + - >- + oc wait -n openshift-gitops application openstack-dataplane + --for jsonpath='{.status.sync.status}'=Synced --timeout=600s + - >- + oc wait -n openshift-gitops application openstack-dataplane + --for jsonpath='{.status.health.status}'=Healthy --timeout=600s + - >- + oc wait -n openstack openstackdataplanedeployment dp-deploy-01 + --for condition=Ready --timeout=2400s diff --git a/scenarios/3-nodes-gitops/bootstrap_vars.yml b/scenarios/3-nodes-gitops/bootstrap_vars.yml index b8888467..4253c3a3 100644 --- a/scenarios/3-nodes-gitops/bootstrap_vars.yml +++ b/scenarios/3-nodes-gitops/bootstrap_vars.yml @@ -24,6 +24,9 @@ ovn_k8s_gateway_config_host_routing: true enable_iscsi: true enable_multipath: true +# GitOps configuration +hotstack_git_server_enabled: true + cinder_volume_pvs: - /dev/vdc - /dev/vdd diff --git a/scenarios/3-nodes-gitops/gitops-manifests/README.md b/scenarios/3-nodes-gitops/gitops-manifests/README.md new file mode 100644 index 00000000..d3a0387c --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/README.md @@ -0,0 +1,50 @@ +# OpenStack Manifests for 3-nodes-gitops Scenario + +This directory contains Kustomize configurations for deploying OpenStack on SNO with one compute node. + +## Structure + +``` +manifests/ +├── operators/ # OpenStack operator installation +├── network/ # Network configuration (NNCP, NAD, MetalLB, NetConfig) +├── controlplane/ # OpenStack control plane +└── dataplane/ # OpenStack data plane (compute node) +``` + +## Design Pattern + +Each directory follows the same pattern: + +1. **kustomization.yaml** - References upstream components from `openstack-k8s-operators/gitops` +2. **patches** - Scenario-specific customizations for SNO + 1 compute + +### Upstream Components + +We leverage reusable components from: +- https://github.com/openstack-k8s-operators/gitops/components/argocd/annotations +- https://github.com/openstack-k8s-operators/gitops/components/rhoso/controlplane +- https://github.com/openstack-k8s-operators/gitops/components/rhoso/dataplane + +### Local Customizations + +Patches customize the base components for this scenario: +- Single node OpenShift (SNO) configuration +- Network CIDRs: ctlplane (192.168.122.0/24), internalapi (172.17.0.0/24), storage (172.18.0.0/24), tenant (172.19.0.0/24) +- Single compute node at 192.168.122.100 +- Storage class: lvms-vg1 + +## Testing Locally + +You can test the kustomize builds locally: + +```bash +kustomize build scenarios/3-nodes-gitops/manifests/operators +kustomize build scenarios/3-nodes-gitops/manifests/network +kustomize build scenarios/3-nodes-gitops/manifests/controlplane +kustomize build scenarios/3-nodes-gitops/manifests/dataplane +``` + +## Deployment + +These manifests are deployed via ArgoCD Applications defined in `../gitops/`. diff --git a/scenarios/3-nodes-gitops/gitops-manifests/controlplane/controlplane-sno-patch.yaml b/scenarios/3-nodes-gitops/gitops-manifests/controlplane/controlplane-sno-patch.yaml new file mode 100644 index 00000000..221406ac --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/controlplane/controlplane-sno-patch.yaml @@ -0,0 +1,85 @@ +--- +apiVersion: core.openstack.org/v1beta1 +kind: OpenStackControlPlane +metadata: + name: openstack-control-plane + namespace: openstack +spec: + storageClass: lvms-local-storage + galera: + templates: + openstack: + replicas: 1 + openstack-cell1: + replicas: 1 + rabbitmq: + templates: + rabbitmq: + replicas: 1 + rabbitmq-cell1: + replicas: 1 + memcached: + templates: + memcached: + replicas: 1 + telemetry: + template: + metricStorage: + monitoringStack: + alertingEnabled: true + scrapeInterval: 30s + storage: + strategy: persistent + retention: 24h + persistent: + pvcStorageRequest: 20G + customMonitoringStack: + alertmanagerConfig: + replicas: 1 + prometheusConfig: + replicas: 1 + dns: + template: + replicas: 1 + options: + - key: server + values: + - 192.168.32.254 + cinder: + template: + cinderAPI: + replicas: 1 + glance: + template: + glanceAPIs: + default: + replicas: 1 + keystone: + template: + replicas: 1 + neutron: + template: + replicas: 1 + nova: + template: + apiServiceTemplate: + replicas: 1 + metadataServiceTemplate: + replicas: 1 + schedulerServiceTemplate: + replicas: 1 + placement: + template: + replicas: 1 + ovn: + template: + ovnController: + nicMappings: + datacentre: ospbr + ovnDBCluster: + ovndbcluster-nb: + replicas: 1 + ovndbcluster-sb: + replicas: 1 + ovnNorthd: + replicas: 1 diff --git a/scenarios/3-nodes-gitops/gitops-manifests/controlplane/kustomization.yaml b/scenarios/3-nodes-gitops/gitops-manifests/controlplane/kustomization.yaml new file mode 100644 index 00000000..0bc3c64a --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/controlplane/kustomization.yaml @@ -0,0 +1,10 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +components: + - https://github.com/openstack-k8s-operators/gitops/components/argocd/annotations?ref=main + - https://github.com/openstack-k8s-operators/gitops/components/rhoso/controlplane/controlplane?ref=main + +patches: + - path: controlplane-sno-patch.yaml diff --git a/scenarios/3-nodes-gitops/gitops-manifests/dataplane/dataplane-deployment-patch.yaml b/scenarios/3-nodes-gitops/gitops-manifests/dataplane/dataplane-deployment-patch.yaml new file mode 100644 index 00000000..50442835 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/dataplane/dataplane-deployment-patch.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: dataplane.openstack.org/v1beta1 +kind: OpenStackDataPlaneDeployment +metadata: + name: dp-deploy-01 + namespace: openstack +spec: + nodeSets: + - example-nodeset diff --git a/scenarios/3-nodes-gitops/gitops-manifests/dataplane/dataplane-nodeset-patch.yaml b/scenarios/3-nodes-gitops/gitops-manifests/dataplane/dataplane-nodeset-patch.yaml new file mode 100644 index 00000000..c56f55d9 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/dataplane/dataplane-nodeset-patch.yaml @@ -0,0 +1,34 @@ +--- +apiVersion: dataplane.openstack.org/v1beta1 +kind: OpenStackDataPlaneNodeSet +metadata: + name: example-nodeset + namespace: openstack +spec: + nodeTemplate: + ansible: + ansibleVars: + edpm_sshd_allowed_ranges: + - 192.168.32.254/32 + - 192.168.122.0/24 + ctlplane_dns_nameservers: + - 192.168.122.80 + nodes: + edpm-compute-0: + ansible: + ansibleHost: 192.168.122.100 + hostName: edpm-compute-0 + networks: + - defaultRoute: true + fixedIP: 192.168.122.100 + name: ctlplane + subnetName: subnet1 + - fixedIP: 172.17.0.100 + name: internalapi + subnetName: subnet1 + - fixedIP: 172.18.0.100 + name: storage + subnetName: subnet1 + - fixedIP: 172.19.0.100 + name: tenant + subnetName: subnet1 diff --git a/scenarios/3-nodes-gitops/gitops-manifests/dataplane/kustomization.yaml b/scenarios/3-nodes-gitops/gitops-manifests/dataplane/kustomization.yaml new file mode 100644 index 00000000..e6077717 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/dataplane/kustomization.yaml @@ -0,0 +1,11 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +components: + - https://github.com/openstack-k8s-operators/gitops/components/argocd/annotations?ref=main + - https://github.com/openstack-k8s-operators/gitops/components/rhoso/dataplane?ref=main + +patches: + - path: dataplane-nodeset-patch.yaml + - path: dataplane-deployment-patch.yaml diff --git a/scenarios/3-nodes-gitops/gitops-manifests/network/ipaddresspool-patch.yaml b/scenarios/3-nodes-gitops/gitops-manifests/network/ipaddresspool-patch.yaml new file mode 100644 index 00000000..c923e303 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/network/ipaddresspool-patch.yaml @@ -0,0 +1,36 @@ +--- +apiVersion: metallb.io/v1beta1 +kind: IPAddressPool +metadata: + name: ctlplane + namespace: metallb-system +spec: + addresses: + - 192.168.122.80-192.168.122.90 +--- +apiVersion: metallb.io/v1beta1 +kind: IPAddressPool +metadata: + name: internalapi + namespace: metallb-system +spec: + addresses: + - 172.17.0.80-172.17.0.90 +--- +apiVersion: metallb.io/v1beta1 +kind: IPAddressPool +metadata: + name: storage + namespace: metallb-system +spec: + addresses: + - 172.18.0.80-172.18.0.90 +--- +apiVersion: metallb.io/v1beta1 +kind: IPAddressPool +metadata: + name: tenant + namespace: metallb-system +spec: + addresses: + - 172.19.0.80-172.19.0.90 diff --git a/scenarios/3-nodes-gitops/gitops-manifests/network/kustomization.yaml b/scenarios/3-nodes-gitops/gitops-manifests/network/kustomization.yaml new file mode 100644 index 00000000..fc53a609 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/network/kustomization.yaml @@ -0,0 +1,16 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +components: + - https://github.com/openstack-k8s-operators/gitops/components/argocd/annotations?ref=main + - https://github.com/openstack-k8s-operators/gitops/components/rhoso/controlplane/networking?ref=main + +resources: + - nncp-sno.yaml + - nad-datacentre.yaml + +patches: + - path: nncp-delete-workers.yaml + - path: netconfig-patch.yaml + - path: ipaddresspool-patch.yaml diff --git a/scenarios/3-nodes-gitops/gitops-manifests/network/nad-datacentre.yaml b/scenarios/3-nodes-gitops/gitops-manifests/network/nad-datacentre.yaml new file mode 100644 index 00000000..a74057af --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/network/nad-datacentre.yaml @@ -0,0 +1,15 @@ +--- +apiVersion: k8s.cni.cncf.io/v1 +kind: NetworkAttachmentDefinition +metadata: + name: datacentre + namespace: openstack +spec: + config: | + { + "cniVersion": "0.3.1", + "name": "datacentre", + "type": "bridge", + "bridge": "ospbr", + "ipam": {} + } diff --git a/scenarios/3-nodes-gitops/gitops-manifests/network/netconfig-patch.yaml b/scenarios/3-nodes-gitops/gitops-manifests/network/netconfig-patch.yaml new file mode 100644 index 00000000..e11b5f63 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/network/netconfig-patch.yaml @@ -0,0 +1,44 @@ +--- +apiVersion: network.openstack.org/v1beta1 +kind: NetConfig +metadata: + name: openstacknetconfig + namespace: openstack +spec: + networks: + - name: ctlplane + dnsDomain: ctlplane.openstack.lab + subnets: + - name: subnet1 + allocationRanges: + - end: 192.168.122.150 + start: 192.168.122.100 + cidr: 192.168.122.0/24 + gateway: 192.168.122.80 + - name: internalapi + dnsDomain: internalapi.openstack.lab + subnets: + - name: subnet1 + allocationRanges: + - end: 172.17.0.150 + start: 172.17.0.100 + cidr: 172.17.0.0/24 + vlan: 20 + - name: storage + dnsDomain: storage.openstack.lab + subnets: + - name: subnet1 + allocationRanges: + - end: 172.18.0.150 + start: 172.18.0.100 + cidr: 172.18.0.0/24 + vlan: 21 + - name: tenant + dnsDomain: tenant.openstack.lab + subnets: + - name: subnet1 + allocationRanges: + - end: 172.19.0.150 + start: 172.19.0.100 + cidr: 172.19.0.0/24 + vlan: 22 diff --git a/scenarios/3-nodes-gitops/gitops-manifests/network/nncp-delete-workers.yaml b/scenarios/3-nodes-gitops/gitops-manifests/network/nncp-delete-workers.yaml new file mode 100644 index 00000000..aff5d73a --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/network/nncp-delete-workers.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: + name: osp-enp6s0-worker-0 +$patch: delete +--- +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: + name: osp-enp6s0-worker-1 +$patch: delete +--- +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: + name: osp-enp6s0-worker-2 +$patch: delete diff --git a/scenarios/3-nodes-gitops/gitops-manifests/network/nncp-sno.yaml b/scenarios/3-nodes-gitops/gitops-manifests/network/nncp-sno.yaml new file mode 100644 index 00000000..b26105a6 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/network/nncp-sno.yaml @@ -0,0 +1,91 @@ +--- +apiVersion: nmstate.io/v1 +kind: NodeNetworkConfigurationPolicy +metadata: + name: master-0 + namespace: openstack + labels: + osp/nncm-config-type: standard +spec: + nodeSelector: + node-role.kubernetes.io/master: "" + desiredState: + dns-resolver: + config: + search: [] + server: + - 192.168.32.254 + interfaces: + - description: internalapi vlan interface + ipv4: + address: + - ip: 172.17.0.10 + prefix-length: 24 + dhcp: false + enabled: true + ipv6: + enabled: false + mtu: 1442 + name: internalapi + state: up + type: vlan + vlan: + base-iface: eth1 + id: 20 + - description: storage vlan interface + ipv4: + address: + - ip: 172.18.0.10 + prefix-length: 24 + dhcp: false + enabled: true + ipv6: + enabled: false + mtu: 1442 + name: storage + state: up + type: vlan + vlan: + base-iface: eth1 + id: 21 + - description: tenant vlan interface + ipv4: + address: + - ip: 172.19.0.10 + prefix-length: 24 + dhcp: false + enabled: true + ipv6: + enabled: false + mtu: 1442 + name: tenant + state: up + type: vlan + vlan: + base-iface: eth1 + id: 22 + - description: ctlplane interface + mtu: 1442 + name: eth1 + state: up + type: ethernet + - bridge: + options: + stp: + enabled: false + port: + - name: eth1 + vlan: {} + description: linux-bridge over ctlplane interface + ipv4: + address: + - ip: 192.168.122.10 + prefix-length: 24 + dhcp: false + enabled: true + ipv6: + enabled: false + mtu: 1442 + name: ospbr + state: up + type: linux-bridge diff --git a/scenarios/3-nodes-gitops/gitops-manifests/operators/kustomization.yaml b/scenarios/3-nodes-gitops/gitops-manifests/operators/kustomization.yaml new file mode 100644 index 00000000..1423628d --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/operators/kustomization.yaml @@ -0,0 +1,8 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +resources: + - namespaces.yaml + - subscription.yaml + - openstack-cr.yaml diff --git a/scenarios/3-nodes-gitops/gitops-manifests/operators/namespaces.yaml b/scenarios/3-nodes-gitops/gitops-manifests/operators/namespaces.yaml new file mode 100644 index 00000000..268b37c7 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/operators/namespaces.yaml @@ -0,0 +1,16 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: openstack-operators + labels: + pod-security.kubernetes.io/enforce: privileged + security.openshift.io/scc.podSecurityLabelSync: "false" +--- +apiVersion: v1 +kind: Namespace +metadata: + name: openstack + labels: + pod-security.kubernetes.io/enforce: privileged + security.openshift.io/scc.podSecurityLabelSync: "false" diff --git a/scenarios/3-nodes-gitops/gitops-manifests/operators/openstack-cr.yaml b/scenarios/3-nodes-gitops/gitops-manifests/operators/openstack-cr.yaml new file mode 100644 index 00000000..336a032c --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/operators/openstack-cr.yaml @@ -0,0 +1,10 @@ +--- +apiVersion: operator.openstack.org/v1beta1 +kind: OpenStack +metadata: + name: openstack + namespace: openstack-operators + annotations: + argocd.argoproj.io/sync-wave: "2" + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +spec: {} diff --git a/scenarios/3-nodes-gitops/gitops-manifests/operators/subscription.yaml b/scenarios/3-nodes-gitops/gitops-manifests/operators/subscription.yaml new file mode 100644 index 00000000..20986a1e --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/operators/subscription.yaml @@ -0,0 +1,22 @@ +--- +apiVersion: operators.coreos.com/v1 +kind: OperatorGroup +metadata: + name: openstack + namespace: openstack-operators + annotations: + argocd.argoproj.io/sync-wave: "1" +--- +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: openstack-operator + namespace: openstack-operators + annotations: + argocd.argoproj.io/sync-wave: "1" +spec: + name: openstack-operator + channel: stable-v1.0 + installPlanApproval: Automatic + source: redhat-operators + sourceNamespace: openshift-marketplace diff --git a/scenarios/3-nodes-gitops/gitops-manifests/secrets/dataplane-ssh-secret.yaml b/scenarios/3-nodes-gitops/gitops-manifests/secrets/dataplane-ssh-secret.yaml new file mode 100644 index 00000000..852dcc09 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/secrets/dataplane-ssh-secret.yaml @@ -0,0 +1,17 @@ +--- +# NOTE: This file is a placeholder example only. +# The actual secret is created dynamically by the automation during deployment +# using the SSH keys from the controller node (/home/zuul/.ssh/id_rsa). +# +# This example shows the expected structure of the secret. +apiVersion: v1 +kind: Secret +metadata: + name: dataplane-ansible-ssh-private-key-secret + namespace: openstack +type: Opaque +stringData: + ssh-privatekey: | + # Placeholder - actual key is generated by automation + ssh-publickey: | + # Placeholder - actual key is generated by automation diff --git a/scenarios/3-nodes-gitops/gitops-manifests/secrets/kustomization.yaml b/scenarios/3-nodes-gitops/gitops-manifests/secrets/kustomization.yaml new file mode 100644 index 00000000..5711eed2 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/secrets/kustomization.yaml @@ -0,0 +1,10 @@ +--- +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization + +namespace: openstack + +resources: + - osp-secret.yaml + - libvirt-secret.yaml + - dataplane-ssh-secret.yaml diff --git a/scenarios/3-nodes-gitops/gitops-manifests/secrets/libvirt-secret.yaml b/scenarios/3-nodes-gitops/gitops-manifests/secrets/libvirt-secret.yaml new file mode 100644 index 00000000..7f033f05 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/secrets/libvirt-secret.yaml @@ -0,0 +1,9 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: libvirt-secret + namespace: openstack +type: Opaque +data: + LibvirtPassword: MTIzNDU2Nzg= diff --git a/scenarios/3-nodes-gitops/gitops-manifests/secrets/osp-secret.yaml b/scenarios/3-nodes-gitops/gitops-manifests/secrets/osp-secret.yaml new file mode 100644 index 00000000..1ecabc73 --- /dev/null +++ b/scenarios/3-nodes-gitops/gitops-manifests/secrets/osp-secret.yaml @@ -0,0 +1,46 @@ +--- +apiVersion: v1 +kind: Secret +metadata: + name: osp-secret + namespace: openstack +type: Opaque +data: + AdminPassword: MTIzNDU2Nzg= + AodhDatabasePassword: MTIzNDU2Nzg= + AodhPassword: MTIzNDU2Nzg= + BarbicanDatabasePassword: MTIzNDU2Nzg= + BarbicanPassword: MTIzNDU2Nzg= + BarbicanSimpleCryptoKEK: c0VGbWRGakRVcVJNMlZlbVlzbFY1eUdOV2pva2lvSlhzZzhOcmxjM2RyVT0= + CeilometerPassword: MTIzNDU2Nzg= + CinderDatabasePassword: MTIzNDU2Nzg= + CinderPassword: MTIzNDU2Nzg= + DatabasePassword: MTIzNDU2Nzg= + DbRootPassword: MTIzNDU2Nzg= + DesignateDatabasePassword: MTIzNDU2Nzg= + DesignatePassword: MTIzNDU2Nzg= + GlanceDatabasePassword: MTIzNDU2Nzg= + GlancePassword: MTIzNDU2Nzg= + HeatAuthEncryptionKey: NzY3YzNlZDA1NmNiYWEzYjlkZmVkYjhjNmY4MjViZjA= + HeatDatabasePassword: MTIzNDU2Nzg= + HeatPassword: MTIzNDU2Nzg= + IronicDatabasePassword: MTIzNDU2Nzg= + IronicInspectorDatabasePassword: MTIzNDU2Nzg= + IronicInspectorPassword: MTIzNDU2Nzg= + IronicPassword: MTIzNDU2Nzg= + KeystoneDatabasePassword: MTIzNDU2Nzg= + ManilaDatabasePassword: MTIzNDU2Nzg= + ManilaPassword: MTIzNDU2Nzg= + MetadataSecret: MTIzNDU2Nzg0Mg== + NeutronDatabasePassword: MTIzNDU2Nzg= + NeutronPassword: MTIzNDU2Nzg= + NovaAPIDatabasePassword: MTIzNDU2Nzg= + NovaCell0DatabasePassword: MTIzNDU2Nzg= + NovaCell1DatabasePassword: MTIzNDU2Nzg= + NovaPassword: MTIzNDU2Nzg= + OctaviaDatabasePassword: MTIzNDU2Nzg= + OctaviaHeartbeatKey: MTIzNDU2Nzg= + OctaviaPassword: MTIzNDU2Nzg= + PlacementDatabasePassword: MTIzNDU2Nzg= + PlacementPassword: MTIzNDU2Nzg= + SwiftPassword: MTIzNDU2Nzg= diff --git a/zuul.d/jobs.yaml b/zuul.d/jobs.yaml index 5899260a..5fd8c6bd 100644 --- a/zuul.d/jobs.yaml +++ b/zuul.d/jobs.yaml @@ -167,3 +167,17 @@ scenario: multi-nodeset files: - ^scenarios/multi-nodeset/.* + +- job: + name: vexxhost-hotstack-3-nodes-gitops + parent: base-hotstack-vexxhost + nodeset: hotstack-image-vexxhost + description: | + Hotstack scenario: 3-nodes-gitops (ArgoCD-based GitOps deployment) + timeout: 10800 + attempts: 1 + vars: + scenario: 3-nodes-gitops + hotstack_git_server_enabled: true + files: + - ^scenarios/3-nodes-gitops/.* diff --git a/zuul.d/projects.yaml b/zuul.d/projects.yaml index 0d07f985..3f07b102 100644 --- a/zuul.d/projects.yaml +++ b/zuul.d/projects.yaml @@ -13,4 +13,5 @@ - vexxhost-hotstack-sno-2-bm - vexxhost-hotstack-3-nodes - vexxhost-hotstack-3-nodes-update + - vexxhost-hotstack-3-nodes-gitops - vexxhost-hotstack-multi-nodeset