From 005e8a562b235191a9c7cf7896240e4aa6132662 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Ravier?= Date: Tue, 24 Mar 2026 11:29:03 +0100 Subject: [PATCH] OCPBUGS-64841: Ensure 'containers' user & group are part of the image This is a combinaison of multiple things: - In [1], the cri-o package has been updated to use systemd-sysusers config instead of using useradd/usermod commands directly. - Starting with OCP 4.19, we've split the OCP packages (here cri-o) from the base RHEL image to the Node image layer. This means that the sysusers scriplet in `%pre` is now called during the node layer build and does not add the user/group to the `/usr/lib/passwd|group` files but to the `/etc/passwd|group` ones. As it does not take into account the existing users & groups from `/usr/lib/passwd|group`, the new `containers` user/group have a UID/GID that collide with an existing user/group. Changes to the `/etc/passwd|group` files are also not propagated to the system ones on updates as those files are changed on first boot as the `core` user is created on the system and thus ostree does not update them anymore. See [2] & [3]. - Starting with OCP 4.19, new nodes start with no `containers` user/group defined (either in `/usr/` or `/etc`) and those are thus created in `/etc` after the switch to the node image, so everything appear to be OK when you create a fresh cluster. Clusters updating to OCP 4.19 with older nodes that used to have the `containers` user/group defined in `/usr/lib/passwd|group` will now no longer have them there and thus systemd-sysusers will attempt to create them on the system. This will however fail as entries for those user/group are left in the `/etc/shadow` and `/etc/gshadow` files. This is [4] but "reversed". The proposed solution here is to keep the `containers` user/group properly defined in the container image in the `/usr/lib/passwd|group` files. Older nodes will thus use those user/group like they used to. New nodes will stop trying to create them. They will have missing `shadow|gshadow` entries however until we fix [4] but that should be an issue as those are not used for interactive/login session users. The medium/longer term fix is to complete the transition away from nss-altfiles for all Bootable Container systems. [1] https://pkgs.devel.redhat.com/cgit/rpms/cri-o/commit/?h=rhaos-4.18-rhel-9&id=240a1e3db29a1d1c1b58dfae1325a9f19c663b91 [2] https://bootc-dev.github.io/bootc/building/users-and-groups.html#system-users-and-groups-added-via-packages-etc [3] https://bootc-dev.github.io/bootc/building/users-and-groups.html#nss-altfiles [4] https://github.com/bootc-dev/bootc/issues/1179 Fixes: https://redhat.atlassian.net/browse/OCPBUGS-64841 Related: https://gitlab.com/fedora/bootc/tracker/-/work_items/76 --- packages-openshift.yaml | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/packages-openshift.yaml b/packages-openshift.yaml index 00057c79..a4f24e13 100644 --- a/packages-openshift.yaml +++ b/packages-openshift.yaml @@ -130,6 +130,38 @@ postprocess: fi fi + - | + #!/usr/bin/env bash + set -xeo pipefail + # Ensure that the containers user & group are created as part of the image. + # We can not move users/groups from the image to dynamically created ones + # until we fix https://github.com/bootc-dev/bootc/issues/1179. + # See https://redhat.atlassian.net/browse/OCPBUGS-64841 and commit message + # for the full details. + + # Only do that when doing a container build + if [[ -f /run/.containerenv ]] && [[ -f /usr/lib/sysusers.d/crio.conf ]]; then + # First, cleanup the broken entries from /etc/passwd|group|shadow|gshadow + sed -i "/^containers:/d" /etc/{passwd,group,shadow,gshadow} + + # We're running as part of a derivation; `systemd-sysusers` will not work + # because it doesn't go through NSS. Hackily put the /usr/lib files in /etc + # temporarily then put them back. + mv /etc/passwd /etc/passwd.bak + mv /etc/group /etc/group.bak + mv /usr/lib/passwd /etc/passwd + mv /usr/lib/group /etc/group + + # Re-create the user/group/shadow/gshadow entries + systemd-sysusers crio.conf + + # Put everything back in place + mv /etc/passwd /usr/lib/passwd + mv /etc/group /usr/lib/group + mv /etc/passwd.bak /etc/passwd + mv /etc/group.bak /etc/group + fi + - | #!/usr/bin/env bash set -xeuo pipefail