diff --git a/.devcontainer/README.md b/.devcontainer/README.md index 7e886a6..61006c8 100644 --- a/.devcontainer/README.md +++ b/.devcontainer/README.md @@ -5,7 +5,7 @@ This devcontainer provides a complete development environment for the Galactic m ## Features ### Languages & Runtimes -- **Go 1.24.2** - For operator, agent, and CNI development +- **Go 1.24.2** - For agent and CNI development ### Kubernetes Tools - **kubectl** - Kubernetes CLI @@ -82,11 +82,10 @@ These are required for: The `post-create.sh` script automatically: 1. Installs Go development tools (gopls, delve, golangci-lint) -2. Installs Kubernetes tools (controller-gen, kustomize, setup-envtest, kind) +2. Installs Kubernetes tools (kustomize, kind) 3. Installs Protocol Buffer compiler and Go plugins 4. Installs network diagnostic tools -5. Generates Kubernetes manifests and DeepCopy methods -7. Configures git safe directory +5. Configures git safe directory ## Getting Started @@ -99,12 +98,6 @@ make build # Run unit tests make test -# Run E2E tests (creates a Kind cluster) -make test-e2e - -# Run the operator locally -make run-operator - # Run the agent locally make run-agent diff --git a/AGENTS.md b/AGENTS.md index 7547ebf..65c678e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,6 +10,7 @@ Galactic is the SRv6 data plane for multi-cloud VPC networking. It consists of a - VPC identifiers are 48-bit hex; VPCAttachment identifiers are 16-bit hex. These are embedded into IPv6 SRv6 endpoint addresses for deterministic route lookups. Both are supplied by an external operator via the CNI config. - Identifiers are also Base62-encoded for interface naming (VRF: `vrfX-Y`, veth host side: `galX-Y`) to keep kernel interface name length within limits. - The binary auto-detects CNI mode via the `CNI_COMMAND` env var; otherwise runs as a Cobra CLI with `agent`, `cni`, and `version` subcommands. +- The Kubernetes operator, VPC/VPCAttachment CRDs, and webhook code have been removed from this repository. They live in a separate companion operator project. ## Tech Stack @@ -43,6 +44,15 @@ Summary: - **Known debt:** Agent and CNI kernel-path code (`internal/agent/srv6/`, `internal/cni/`) has no unit coverage; these paths are best covered by integration or e2e tests. Only `pkg/common/util` has unit test coverage. - **In flux:** The SRv6 route management (`internal/agent/srv6/`) and VRF utilities (`pkg/common/vrf/`) are the least tested and most likely to change as multi-cloud routing matures. BGP integration is in progress. +## Lab Environments + +Two ContainerLab-based environments live under `lab/`: + +- **`lab/network/`** — Standalone SRv6 underlay lab. Eight FRR + GoBGP nodes across PE, transit, and route-reflector roles. Use to develop and test BGP/SRv6 routing behaviour independently of Kubernetes. +- **`lab/gvpc/`** — Three Kind clusters (iad, sjc, infra) wired over an SRv6 transit mesh. FRR runs as a hostNetwork DaemonSet on each cluster's worker for the eBGP underlay; GoBGP on iad and sjc workers handles L3VPN type-5 routes over iBGP to the infra route reflector. + +See `lab/README.md` for quick-start commands and prerequisites for each environment. + ## New Developer Entry Points 1. Run `make build` to verify toolchain; run `make test` to confirm unit tests pass. @@ -53,3 +63,4 @@ Summary: **Likely trip-ups:** - `make run-agent` requires elevated privileges (netlink, VRF, SRv6 operations need `CAP_NET_ADMIN`). +- There is no operator or webhook in this repository; those components are in a separate project. diff --git a/Makefile b/Makefile index 9fadaf1..9d2fee6 100644 --- a/Makefile +++ b/Makefile @@ -53,11 +53,7 @@ vet: ## Run go vet against code. test: fmt vet ## Run tests. GOOS=linux go test $$(go list ./... | grep -v /e2e) -coverprofile cover.out -# TODO(user): To use a different vendor for e2e tests, modify the setup under 'tests/e2e'. -# The default setup assumes Kind is pre-installed and builds/loads the Manager Docker image locally. -# CertManager is installed by default; skip with: -# - CERT_MANAGER_INSTALL_SKIP=true -KIND_CLUSTER ?= galactic-test-e2e +KIND_CLUSTER ?= galactic-e2e .PHONY: setup-test-e2e setup-test-e2e: ## Set up a Kind cluster for e2e tests if it does not exist @@ -73,11 +69,6 @@ setup-test-e2e: ## Set up a Kind cluster for e2e tests if it does not exist $(KIND) create cluster --name $(KIND_CLUSTER) ;; \ esac -.PHONY: test-e2e -test-e2e: setup-test-e2e fmt vet ## Run the e2e tests. Expected an isolated environment using Kind. - KIND_CLUSTER=$(KIND_CLUSTER) go test ./test/e2e/ -v -ginkgo.v - $(MAKE) cleanup-test-e2e - .PHONY: cleanup-test-e2e cleanup-test-e2e: ## Tear down the Kind cluster used for e2e tests @$(KIND) delete cluster --name $(KIND_CLUSTER) @@ -108,7 +99,7 @@ build: fmt vet ## Build galactic binary. run-agent: fmt vet ## Run agent from your host. go run ./cmd/galactic/main.go agent -# If you wish to build the manager image targeting other platforms you can use the --platform flag. +# If you wish to build the image targeting other platforms you can use the --platform flag. # (i.e. docker build --platform linux/arm64). However, you must enable docker buildKit for it. # More info: https://docs.docker.com/develop/develop-images/build_enhancements/ .PHONY: docker-build @@ -119,13 +110,10 @@ docker-build: ## Build docker image with the unified binary. docker-push: ## Push docker image with the unified binary. $(CONTAINER_TOOL) push ${IMG} -# PLATFORMS defines the target platforms for the manager image be built to provide support to multiple -# architectures. (i.e. make docker-buildx IMG=myregistry/mypoperator:0.0.1). To use this option you need to: -# - be able to use docker buildx. More info: https://docs.docker.com/build/buildx/ -# - have enabled BuildKit. More info: https://docs.docker.com/develop/develop-images/build_enhancements/ -# - be able to push the image to your registry (i.e. if you do not set a valid value via IMG=> then the export will fail) -# To adequately provide solutions that are compatible with multiple platforms, you should consider using this option. -PLATFORMS ?= linux/arm64,linux/amd64,linux/s390x,linux/ppc64le +# PLATFORMS defines the target platforms for the image build. Requires docker buildx and BuildKit. +# Usage: make docker-buildx IMG=myregistry/galactic:0.0.1 +# Requires IMG to be set to a valid registry/tag or the push will fail. +PLATFORMS ?= linux/amd64,linux/arm64 .PHONY: docker-buildx docker-buildx: ## Build and push docker image for the unified binary for cross-platform support # copy existing Dockerfile and insert --platform=${BUILDPLATFORM} into Dockerfile.cross, and preserve the original Dockerfile diff --git a/README.md b/README.md index 5322ba6..396dc82 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,12 @@ Under the hood, Galactic uses Segment Routing over IPv6 (SRv6) for efficient, de ## Getting Started -See the [`lab/`](./lab/) directory for example topologies and the [DevContainer](./.devcontainer/) for development environment setup. +Two ContainerLab environments are available under [`lab/`](./lab/): + +- **[`lab/network/`](./lab/network/)** — Standalone SRv6 underlay lab (FRR + GoBGP, no Kubernetes). Good starting point for understanding the routing layer. +- **[`lab/gvpc/`](./lab/gvpc/)** — Three Kind clusters wired over an SRv6 transit mesh. The full GVPC multi-cluster environment. + +See the [DevContainer](./.devcontainer/) for development environment setup. ## License diff --git a/lab/README.md b/lab/README.md index 2aa95ed..ed62ec0 100644 --- a/lab/README.md +++ b/lab/README.md @@ -4,9 +4,8 @@ Local development and integration-testing environments for [Galactic VPC](https: ``` lab/ -├── network/ # ContainerLab SRv6 underlay network -└── containers/ - └── kindest-node-galactic/ # Custom Kind node image with Galactic pre-installed +├── network/ # ContainerLab SRv6 underlay network (standalone) +└── gvpc/ # ContainerLab GVPC multi-cluster lab (3 Kind clusters over SRv6 mesh) ``` --- @@ -38,74 +37,29 @@ make down # tear down --- -## `containers/kindest-node-galactic/` — Custom Kind Node Image +## `gvpc/` — GVPC Multi-Cluster Lab -A `kindest/node` image extended with the tooling and Kubernetes manifests needed to -run a full Galactic stack inside a [Kind](https://kind.sigs.k8s.io/) cluster. +Three Kind clusters (iad, sjc, infra) connected over an IPv6 SRv6 transit mesh. Each +cluster runs FRR as a node routing daemon (hostNetwork DaemonSet) to peer with the +transit layer via BGP unnumbered. GoBGP runs alongside FRR on the iad and sjc workers +to exchange L3VPN type-5 routes with the infra route reflector over iBGP. -``` -kindest-node-galactic/ -├── Dockerfile -├── resources/ # Kubernetes manifests applied at cluster boot -│ ├── agent.k8s.yaml -│ └── operator.k8s.yaml -└── scripts/ - └── install.sh # Installs Cilium, cert-manager, Multus, and Galactic -``` - -`install.sh` is invoked once per node after the cluster comes up. On the control-plane -node it applies each Kubernetes manifest in order (Cilium → cert-manager → Multus → -Galactic operator → agent). On worker nodes it loads kernel modules, -sets SRv6 sysctls, and drops in the CNI binaries. +See [`gvpc/README.md`](gvpc/README.md) for topology details, addressing, and +verification commands. ### Prerequisites +- ContainerLab ≥ 0.54 - Docker -- [`kind`](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) -- Linux kernel with `vrf` module and SRv6 support (or a VM with those features) - -### Build - -```bash -cd containers/kindest-node-galactic -docker build -t kindest/node:galactic . -``` +- `kind` CLI +- Host kernel with SRv6 support -### Use with Kind - -Reference the custom image in your Kind cluster config: - -```yaml -# kind-config.yaml -kind: Cluster -apiVersion: kind.x-k8s.io/v1alpha4 -nodes: - - role: control-plane - image: kindest/node:galactic - - role: worker - image: kindest/node:galactic - - role: worker - image: kindest/node:galactic -``` - -```bash -kind create cluster --config kind-config.yaml -``` - -After the cluster is up, run `install.sh` on each node: +### Quick start ```bash -for node in $(kind get nodes); do - docker exec "$node" /galactic/scripts/install.sh -done +cd gvpc +make up # build Kind node image, apply host sysctls, deploy lab +make underlay # apply FRR DaemonSets to all three clusters +make overlay # apply GoBGP DaemonSets to iad and sjc clusters +make down # tear down ``` - -### Component versions (pinned in `scripts/install.sh`) - -| Component | Version | -|--------------|----------| -| Cilium CLI | v0.18.8 | -| cert-manager | v1.19.1 | -| Multus CNI | v4.2.3 | -| CNI plugins | v1.8.0 | -| Galactic | v0.0.5 |