Remove unneeded packages from the Collector image.

[ovalenti]

Description

Our base image contains some packages that are not needed by Collector. This PR introduces 'orphaner', a tool able to use the RPM dependency graph to list all packages that are not part of Collector deps. Then we uninstall them.

We don't use the available python/perl tools because they have dependencies themselves.
'orphaner' uses only librpm.

Checklist

• Investigated and inspected CI test results
• Updated documentation accordingly

Automated testing

• Added unit tests
• Added integration tests
• Added regression tests

Testing done
With this change, on ubi9, the Collector image archive decreases size from 106MB to 92MB.
Notable libraries going away:

• libbz2
• libblkid
• gio / glib / gmodule / gobject
• gmp
• gnutls
• gss / krb5
• json-c
• smartcols
• systemd
• udev
• xml2
• and some fonts

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 27.38%. Comparing base (a9ed8fe) to head (9ab7fd2).
⚠️ Report is 2 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2970   +/-   ##
=======================================
  Coverage   27.38%   27.38%           
=======================================
  Files          95       95           
  Lines        5427     5427           
  Branches     2548     2548           
=======================================
  Hits         1486     1486           
  Misses       3214     3214           
  Partials      727      727           

Flag	Coverage Δ
collector-unit-tests	27.38% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[msugakov]

This is a notable effort, but I have to tell there's another way. Instead of starting from ubi-minimal and removing packages, you can start from ubi-micro which is much slimmer base (it can't get any slimmer for the time being) and install only necessary packages.

It's a tiny bit involved how to install RPMs into ubi-micro, but solvable. Would you be interested to try that out?

[ovalenti]

Thanks for this interesting suggestion.

I put some thought into this possibility, and for upstream, we don't have a mean to install packages, so we rely on the packages already in the base image. In particular, ubi-micro doesn't include elfutils-libelf. We would need to build those dependencies ourselves, which starts to look like buildroot :/

For downstream, we are installing supplementary packages already, so ubi-micro can be considered. However, it would make the 2 pipelines diverge a bit more.

It's a tiny bit involved how to install RPMs into ubi-micro, but solvable. Would you be interested to try that out?

How much "involved" is it ?

[msugakov]

See the Dockerfile in the Example section here.

The flow in words:

1. Take contents of ubi-micro as files and place them somewhere in /mnt/... of ubi(-normal).
2. Use dnf --installroot=/mnt/... to install packages there.
3. Create a stage which starts from ubi-micro and overwrite its contents with /mnt/... of that ubi(-normal) stage.

The idea is that ubi-micro lacks any *dnf, but ubi(-normal) can install packages in arbitrary paths. That path can be a contents of ubi-micro image.

As folks state in the parent doc, for FIPS you'll need to install openssl openssl-libs, and, I believe, ca-certificates in addition to whatever other RPMs that you need.

You mentioned the different situation between upstream and downstream. Is it because something upstream is CentOS-based?

[ovalenti]

Thanks for the links. I can see now what it would take to move to this strategy.

You mentioned the different situation between upstream and downstream. Is it because something upstream is CentOS-based?

In upstream, we build the collector binary using centos. Then we copy it in a ubi container.

This all comes from the fact that we cannot install packages in a ubi container, and also that the team prefers to keep downstream & upstream as close as possible (I tend to be of a different opinion on that point).

[msugakov]

I suppose CentOS is used upstream because you need some RHEL (entitled) packages for building, right?
You don't need RHEL packages for running collector, do you?

Yes, it's harder to get RHEL packages upstream and it's much easier to use CentOS instead. That's fine.

Though, the final stage could be the same in both upstream and downstream. It could be based on ubi-micro in both cases and could be provisioned identically.

Upstream you would: build on CentOS, run on ubi-micro.
Downstream you would: build on UBI with the help of entitled RPMs, run also on ubi-micro.

How does that sound?