feat: migrate to testcontainers-go v0.42.0 and upgrade dependencies by wu-sheng · Pull Request #146 · apache/skywalking-infra-e2e

wu-sheng · 2026-04-14T02:10:16Z

Summary

Major dependency upgrade and code simplification:

Migrate testcontainers-go from v0.11.1 to v0.42.0, using native compose/container APIs
Delete 686 lines of custom DockerContainer/DockerProvider re-implementation (compose_provider.go, compose_listener.go)
Upgrade all major dependencies: docker v28.5.2, k8s v0.35.3, kind v0.31.0, cobra v1.10.2
Remove docker-compose v1 binary install from CI — v0.42.0 uses Docker Compose v2 plugin natively
Add multi-platform CI build matrix (ubuntu/windows/macos) and enable CGO for native builds

What Changed

File	Change
`compose_provider.go`	Deleted (601 lines) — native testcontainers DockerContainer replaces custom impl
`compose_listener.go`	Deleted (85 lines) — `stack.ServiceContainer()` replaces Docker event listening
`compose.go`	Rewritten: `LocalDockerCompose` → `compose.NewDockerComposeWith` + `stack.Up/ServiceContainer`
`cleanup/compose.go`	Rewritten: uses `stack.Down` with `RemoveVolumes`/`RemoveOrphans`
`kind.go`	Updated docker image API types (`types.ImageListOptions` → `image.ListOptions`)
`k8s.go`	Updated `NewShortcutExpander` to use warning callback (new k8s API)
`action.yaml`	Updated Go to 1.26, removed docker-compose v1 install
`e2e-test.yaml`	Removed docker-compose v1 install, added CGO_ENABLED=1
`build.yaml`	Split into lint-and-test + build-platform matrix (ubuntu/windows/macos)
`Makefile`	`make build` builds for current OS only, added CGO_ENABLED per platform
`Dockerfile`	Added build-essential for CGO, CGO_ENABLED=1 for linux build

User-Facing Contract Preserved

Config format (e2e.yaml) unchanged
Environment variable export: <SERVICE>_host, <SERVICE>_<PORT>
Log files at logDir/serviceName/std.log
CLI flags and subcommands unchanged
Cleanup behavior unchanged
GITHUB_RUN_ID project identifier unchanged

Test plan

go build ./... passes
make lint passes (0 issues)
go test ./... passes (all unit tests)
make build passes locally (macOS with CGO)
Local compose e2e test: setup + cleanup work correctly
CI Lint and Test passes
CI Build (ubuntu/windows/macos) passes
CI E2E Test (compose + kind) passes

Supersedes #139

🤖 Generated with Claude Code

…ndencies - Delete compose_provider.go (601 lines) — custom DockerContainer/DockerProvider re-implementation replaced by native testcontainers-go APIs - Delete compose_listener.go (85 lines) — Docker event listener no longer needed; containers are accessible after stack.Up() via stack.ServiceContainer() - Rewrite compose.go to use compose.NewDockerComposeWith + stack.Up/ServiceContainer - Rewrite cleanup/compose.go to use stack.Down - Upgrade testcontainers-go v0.11.1 → v0.42.0 - Upgrade docker/docker v20.10.7 → v28.5.2 - Upgrade k8s.io/* v0.22.2 → v0.35.3 - Upgrade sigs.k8s.io/kind v0.27.0 → v0.31.0 - Upgrade spf13/cobra v1.8.0 → v1.10.2 - Remove docker-compose v1 binary install from CI (v0.42.0 uses Docker Compose v2 plugin) - Update action.yaml Go version to 1.26 Closes #139 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Split Build workflow into lint-and-test + build-platform (ubuntu/windows/macos) matrix jobs. Enable CGO for native builds. Update Dockerfile with build-essential for CGO. Change `make build` to build for current OS only (CI matrix handles cross-platform). Based on #139 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…pose version field Pin infra-e2e to apache/skywalking-infra-e2e#146 (81adeec) which migrates to testcontainers-go v0.42.0, using Docker Compose v2 plugin natively and removing docker-compose v1 dependency. Remove the deprecated `version` field from all docker-compose files for Compose v2 compatibility.

compose.Wait(true) maps to `docker compose up --wait` which requires ALL containers to remain running and healthy. This is too strict for e2e tests where some containers may not have healthchecks or may exit after init. Port readiness is already handled by WaitForService() with per-port wait strategies. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

wu-sheng · 2026-04-14T03:24:08Z

Fixed: removed compose.Wait(true) which mapped to docker compose up --wait, requiring all containers to reach running+healthy state. This was too strict — containers without healthchecks or init containers that exit would cause failures.

Port readiness is already handled by stack.WaitForService() with per-port wait strategies, which is sufficient.

Found via apache/skywalking#13817 testing.

…pose version field Pin infra-e2e to apache/skywalking-infra-e2e#146 (81adeec) which migrates to testcontainers-go v0.42.0, using Docker Compose v2 plugin natively and removing docker-compose v1 dependency. Remove the deprecated `version` field from all docker-compose files for Compose v2 compatibility.

No longer needed — testcontainers-go v0.42.0 uses Docker Compose v2 plugin which fully supports the containerd image store. Reverts the workaround added in 8c21e43 (#140). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

WaitForService strategies only execute when compose.Wait(true) is passed, but compose.Wait(true) is too strict (requires all containers healthy). Instead, manually wait for each port via TCP dial after Up() returns, matching the old code's behavior of waitPortUntilReady(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Docker SDK v28.5.2 defaults to API v1.51, but GitHub Actions runners may only support up to v1.48. Set DOCKER_API_VERSION to the daemon's supported version to ensure compatibility. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add INFO-level logs for port wait start/completion and DEBUG-level for retry attempts. Also try connecting immediately before the first ticker interval to avoid unnecessary 500ms delay when ports are already ready. Log when a service has no ports to wait for. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

After stack.Up(), wait for each container's Docker healthcheck to pass (if defined) before doing TCP port checks. This matches the old behavior where docker-compose up -d would respect depends_on: condition: service_healthy. Also restored internal container port check (/proc/net/tcp + nc + /dev/tcp) from the old WaitPort implementation to align with original behavior. The wait sequence is now: 1. stack.Up(ctx) — start containers 2. waitForHealthy — poll Docker health status until healthy (if healthcheck defined) 3. waitForPort — external TCP dial + internal container check 4. Export env vars Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The old code only waited for services that had ports: in docker-compose.yml. Services without ports were completely skipped. The healthcheck wait was incorrectly running on ALL services, causing timeouts on services like init containers or senders whose healthchecks depend on other services. Now: services without ports get log streaming only. Services with ports get healthcheck wait + port wait + env export (matching old behavior). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

The old code never checked Docker healthcheck status. It only did TCP dial + internal /proc/net/tcp check. Many compose files use healthchecks with tools (nc) not available in the container image, causing Docker to mark containers as unhealthy even when the application is running fine. Remove waitForHealthy entirely to match old behavior exactly: port-level waiting only, no Docker healthcheck dependency. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

kezhenxu94 · 2026-04-15T02:14:42Z

-        sudo systemctl restart docker
-        docker version
-        docker info
        echo "DOCKER_API_VERSION=$(docker version --format '{{.Server.APIVersion}}')" >> "$GITHUB_ENV"


Do we still need this now? This entire step was added to work around the issue, I think the entire step Set Docker API version can be removed

Yes, we still need this line to keep client version following the server declared.

kezhenxu94 · 2026-04-15T02:16:41Z


 .PHONY: build
-build: windows linux darwin
+build: $(GOOS)


Changing the build target to only build the current OS will break the release target? The release target depends on this to build all binaries for all OS'es

The build target must build all platforms (windows/linux/darwin) because release-bin depends on it. Each platform build uses CGO_FLAG to enable CGO only for native builds. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Each matrix runner (ubuntu/windows/macos) now builds only its own platform instead of all three redundantly. The `make build` target is preserved for release which needs all platforms. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…include

…pose version field (#13817) **Changes:** - Bump `skywalking-infra-e2e` to `530f997` (merged apache/skywalking-infra-e2e#146), which migrates testcontainers-go from v0.11.1 to v0.42.0, using Docker Compose v2 plugin natively and removing docker-compose v1 dependency. - Remove deprecated `version` field from all docker-compose files (Compose v2 ignores it with a warning). **What the infra-e2e change does:** - Migrates testcontainers-go v0.11.1 → v0.42.0, using native `compose.NewDockerComposeWith` API - Deletes ~686 lines of custom `DockerContainer`/`DockerProvider` reimplementation - Removes docker-compose v1 binary installation from CI `action.yaml` - Adds TCP port waiting after `docker compose up -d` (matching old behavior) - Restores `DOCKER_API_VERSION` negotiation for Docker 29+ compatibility - User-facing contract (e2e.yaml format, env vars, log paths, CLI) is preserved

wu-sheng and others added 2 commits April 14, 2026 10:09

wu-sheng added the feature label Apr 14, 2026

wu-sheng added this to the 1.4.0 milestone Apr 14, 2026

wu-sheng mentioned this pull request Apr 14, 2026

Bump infra-e2e to testcontainers-go v0.42.0 and remove deprecated compose version field apache/skywalking#13817

Merged

1 task

wu-sheng and others added 7 commits April 14, 2026 11:50

kezhenxu94 reviewed Apr 15, 2026

View reviewed changes

wu-sheng mentioned this pull request Apr 15, 2026

feat: upgrade dependencies and migrate deprecated APIs to new ones #139

Closed

2 tasks

wu-sheng requested a review from kezhenxu94 April 15, 2026 02:28

kezhenxu94 reviewed Apr 15, 2026

View reviewed changes

Comment thread .github/workflows/build.yaml Outdated

wu-sheng and others added 2 commits April 15, 2026 10:33

ci: simplify build matrix by moving OS-to-target mapping into matrix …

e451463

…include

kezhenxu94 approved these changes Apr 15, 2026

View reviewed changes

kezhenxu94 merged commit 530f997 into main Apr 15, 2026
6 checks passed

kezhenxu94 deleted the feat/migrate-testcontainers-v042 branch April 15, 2026 03:08

wu-sheng mentioned this pull request Apr 16, 2026

fix: disable Ryuk reaper when cleanup.on is never #147

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: migrate to testcontainers-go v0.42.0 and upgrade dependencies#146

feat: migrate to testcontainers-go v0.42.0 and upgrade dependencies#146
kezhenxu94 merged 13 commits intomainfrom
feat/migrate-testcontainers-v042

wu-sheng commented Apr 14, 2026 •

edited

Loading

Uh oh!

wu-sheng commented Apr 14, 2026

Uh oh!

kezhenxu94 Apr 15, 2026

Uh oh!

wu-sheng Apr 15, 2026

Uh oh!

kezhenxu94 Apr 15, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

wu-sheng commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Changed

User-Facing Contract Preserved

Test plan

Uh oh!

wu-sheng commented Apr 14, 2026

Uh oh!

kezhenxu94 Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

wu-sheng Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

kezhenxu94 Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

wu-sheng commented Apr 14, 2026 •

edited

Loading