Building features in a sensible way (without bundlign the entire OCI artifact as a layer requires BuildKit, mainly because we want to build with a context outside the build directory.
I believe upstream has a fallback where it tries to handle no buildkit by instead bundling the entire artifact. This probably isn't a good idea. We would probably prefer to throw an exception in this case (perhaps gate this behind a flag so we can have upstream parity).
Also, podman builds with buildah instead, which comes with support for build context directory out of the box! But I think upstream probably does not handle this well and may fall back on the bad bundle the entire artifact approach instead. Need to check and confirm behaviour here.
ChatGPT context below
The Dev Container CLI has two Feature-content delivery modes:
Build-context mode
Uses a named build context:
--build-context dev_containers_feature_content_source=/tmp/...
and generated Dockerfile lines like:
RUN --mount=type=bind,from=dev_containers_feature_content_source,...
This is the mode you want for a large bundled apt repo.
Non-BuildKit / fallback mode
Builds a temporary image called something like:
dev_container_feature_content_temp
with:
FROM scratch
COPY . /tmp/build-features/
Then the actual devcontainer build copies Feature content from that image into /tmp/dev-container-features/.... That is exactly the bad path for your large artifact, because it can put the entire bundled repo into image layers, and in the current non-BuildKit path it is not removed by the CLI-generated Dockerfile after installation.
The CLI decides this based on whether it thinks it has a sufficiently capable buildx/buildkit path. In the source, if params.buildKitVersion is present, it invokes buildx build and adds --build-context entries; otherwise it invokes plain build. Separately, Feature handling uses build contexts only when the parsed buildkit/buildx version is at least 0.8.0; otherwise it falls back to the temp-image COPY . /tmp/build-features/ approach.
With Podman, this is awkward because podman buildx build is not Docker Buildx. Podman documents podman buildx build as an alias of podman build, implemented through Buildah, and explicitly says not all Docker buildx features are available. However, current Podman/Buildah does support the specific --build-context mechanism, including references from COPY --from=... and RUN --mount=from=....
So I would not state the failure as “Podman cannot do it.” A more accurate statement is:
The Dev Container CLI’s large-Feature-safe path depends on named build contexts. Docker BuildKit/buildx is the reference path. Podman/Buildah may support the required primitives, especially for single-container builds, but this must be explicitly tested because Podman is only Docker-CLI-compatible at the surface, not implementation-equivalent.
There is evidence that the Dev Container CLI does try to use podman buildx version and can route through Podman/Buildah; maintainers have also noted that Buildah versions seen through podman buildx version support --build-context. But there are also known rough edges around Podman + Compose + build contexts, including an open/draft PR titled “Do not use build contexts with Podman and Compose.”
Building features in a sensible way (without bundlign the entire OCI artifact as a layer requires BuildKit, mainly because we want to build with a context outside the build directory.
I believe upstream has a fallback where it tries to handle no buildkit by instead bundling the entire artifact. This probably isn't a good idea. We would probably prefer to throw an exception in this case (perhaps gate this behind a flag so we can have upstream parity).
Also, podman builds with buildah instead, which comes with support for build context directory out of the box! But I think upstream probably does not handle this well and may fall back on the bad bundle the entire artifact approach instead. Need to check and confirm behaviour here.
ChatGPT context below
The Dev Container CLI has two Feature-content delivery modes:
Build-context mode
Uses a named build context:
--build-context dev_containers_feature_content_source=/tmp/...
and generated Dockerfile lines like:
RUN --mount=type=bind,from=dev_containers_feature_content_source,...
This is the mode you want for a large bundled apt repo.
Non-BuildKit / fallback mode
Builds a temporary image called something like:
dev_container_feature_content_temp
with:
FROM scratch
COPY . /tmp/build-features/
Then the actual devcontainer build copies Feature content from that image into /tmp/dev-container-features/.... That is exactly the bad path for your large artifact, because it can put the entire bundled repo into image layers, and in the current non-BuildKit path it is not removed by the CLI-generated Dockerfile after installation.
The CLI decides this based on whether it thinks it has a sufficiently capable buildx/buildkit path. In the source, if params.buildKitVersion is present, it invokes buildx build and adds --build-context entries; otherwise it invokes plain build. Separately, Feature handling uses build contexts only when the parsed buildkit/buildx version is at least 0.8.0; otherwise it falls back to the temp-image COPY . /tmp/build-features/ approach.
With Podman, this is awkward because podman buildx build is not Docker Buildx. Podman documents podman buildx build as an alias of podman build, implemented through Buildah, and explicitly says not all Docker buildx features are available. However, current Podman/Buildah does support the specific --build-context mechanism, including references from COPY --from=... and RUN --mount=from=....
So I would not state the failure as “Podman cannot do it.” A more accurate statement is:
The Dev Container CLI’s large-Feature-safe path depends on named build contexts. Docker BuildKit/buildx is the reference path. Podman/Buildah may support the required primitives, especially for single-container builds, but this must be explicitly tested because Podman is only Docker-CLI-compatible at the surface, not implementation-equivalent.
There is evidence that the Dev Container CLI does try to use podman buildx version and can route through Podman/Buildah; maintainers have also noted that Buildah versions seen through podman buildx version support --build-context. But there are also known rough edges around Podman + Compose + build contexts, including an open/draft PR titled “Do not use build contexts with Podman and Compose.”