From 74439d907020ffc9db6882c9e9f6eb5d3b622381 Mon Sep 17 00:00:00 2001 From: Sam James Date: Sat, 20 Sep 2025 18:07:05 +0100 Subject: [PATCH 1/4] general-concepts/bundled-deps: new section I've tried to faithfully port the wiki page [0] to the devmanual in this commit, and intend to change the contents as required in followups, to allow easier comparison and to retain provenance. [0] https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies Closes: https://bugs.gentoo.org/300625 Signed-off-by: Sam James --- general-concepts/bundled-deps/text.xml | 392 +++++++++++++++++++++++++ general-concepts/text.xml | 1 + 2 files changed, 393 insertions(+) create mode 100644 general-concepts/bundled-deps/text.xml diff --git a/general-concepts/bundled-deps/text.xml b/general-concepts/bundled-deps/text.xml new file mode 100644 index 00000000..2dfaf55f --- /dev/null +++ b/general-concepts/bundled-deps/text.xml @@ -0,0 +1,392 @@ + + + +Bundled dependencies + + +

+The intent of this page is to collect information on dependency bundling +and static linking as a reference to refer upstream developers, instead of +explaining the same thing repeatedly by e-mail. +

+ + +
+When is code bundled? + + +

+Say you develop and distribute a piece of software: a game, a library, anything. +Now, the code is considered bundled if any of the following conditions occur: +

+ +
    +
  • + Statically linking against a system library +
    • +
  • + Shipping and using your own copy of a library +
    • +
  • + Including and (unconditionally) using snippets of code copied from + a library +
    • +
+ +

+In other words, code bundling occurs whenever a program or library ends +up containing code that does not belong to it. +

+ + +
+ +
+Temptations + + +

+There are reasons why bundling dependencies and using static linking occurs; +there are certain benefits to it. So why is it tempting to do such a thing? +

+ + + + +Comforting non-Linux users + + +

+Especially in Windows, shipping dependencies can be a favour to users +to save end users having to manually install dependencies or additional +libraries. Without a package manager, there is no real solution to that on +Windows anyway. +

+ +

+It is tempting when using bundled code on Windows to bundle on GNU/Linux too. +It feels consistent and fits together nicely in the mind of the software +author. +

+ + + + + +Easing up adoption despite odd dependencies + + +

+If a software package P has some dependency D that is not yet +packaged for major distributions, D makes it harder for P to +get in as packaging P forces the new maintainer to package D +him/herself or to wait for someone else to package it for him/her. +

+ +

+Bundling D hides the dependency on D in a way: if the packager +is not paying close attention P may even get in despite and with the +bundled dependency. (It is, however, only a matter of time until someone +notices the bundling.) +

+ + + + + +Private forks + + +

+If P uses a library D, the developers of P may wish +to make some changes to D, for example to add a new feature, modify +the API, or change the default behavior. If the developers of D +for whatever reason are opposed to these changes, the developers of +P may want to fork D. +

+ +

+But publishing and properly maintaining a fork takes time and effort, so +the developers of P could be tempted to take the easy road, bundle +their patched version of D with P, and maybe occasionally +update it for upstream D changes. +

+ + +
+ +
+Problems + + +

+So why is bundling dependencies and static linking bad after all? +

+ + + +Security implications + + +

+Let's consider you're a developer of foo and your foo uses +libbar. +

+ +

+Now, a critical important security flaw has been found in libbar +(say, remote privilege escalation). The problem is large enough that devs +of libbar release a fixed version right away, and distributions package +it quickly to decrease the possibility of break-in to users' systems to a +minimum. +

+ +

+If a particular distribution has an efficient security upgrade system, the +patched library can get there in less than 24 hours. But that would be of +no use to foo users which will still use the earlier vulnerable library. +

+ +

+Now, depending on how bad things are: +

+ +
    +
  • + If foo statically linked against libbar, then the users would + either have to rebuild foo themselves to make it use the fixed library + or distribution developers would have to make a new package for foo and + make sure it gets to user systems along with libbar (assuming they + are aware that the package is statically linked) +
    • +
  • + If foo bundled a local copy of libbar, then they would have to wait + till you discover the vulnerability, update libbar sources, release + the new version and distributions package the new version +
    • +
+ +

+In the meantime, users probably even won't know they are running a vulnerable +application just because they won't know there's a vulnerable library +statically linked into the executables. +

+ +

+Examples: +

+ +
    +
  • + + CVE-2016-3074 has to be + fixed in PHP + (where it is bundled) after it is + + fixed in libgd (upstream) +
    • +
+ + + + +Waste of hardware resources + + +

+Say a media player is bundling library libvorbis. If libvorbis is also +installed system-wide this means that two copies of libvorbis: +

+ +
    +
  1. + occupy twice as much space on disk +
    2. +
  2. + occupy (up to) twice as much RAM (of the page cache) +
    3. +
+ + + + +Waste of development time downstream + + +

+Due to the + +consequences of bundled dependencies, many hours of downstream developer +time are wasted that could have been put to more useful work. +

+ + + + +Potential for symbol collisions + + +

+If a program P uses a system-installed library A and also uses +another library B which bundles library A, there is a potential +for symbol collisions. +

+ +

+This means that P might use an interface, such as my_function() +and that the my_function() symbol would be present in both A +and the version of A bundled inside of library B. +

+ +

+If the system-installed copy of A and the copy of A compiled +into library B are from different releases of library A, then +the operation of the interface my_function() might behave differently +in each copy of A. +

+ +

+Since the program P was compiled against the system-installed copy of +A and for various other reasons, if P ends up using the +my_function() interface from the version of A bundled in +library B instead of the interface in the system-installed copy. +

+ +

+This can potentially result in crashes or strange unpredictable behavior. +

+ +

+This sort of problem can be prevented if library B uses symbol +visibility tricks when it links against library A, which would cause +library B not to export library A's interfaces. +

+ +

+Examples: +

+ +
    +
  • + libmagic bundled with PHP (Gentoo + bug 471682, + PHP bug 66095) +
    • +
+ + +
+ +
+Downstream consequences + + +

+When a bundled dependency is discovered downstream this has a number of +bad consequences. +

+ + + + +Analysis + + +

+So there is a copy of libvorbis bundled with that media player. Which +version is it? Has it been modified? +

+ + + +Separating forks from copies + + +

+Before the bundled dependency can be replaced by the system-widely installed +one, we need to know if it has been modified: we have to know if it's a fork. +

+ +

+If it is a fork it may or may not be replaced without breaking something. +

+ +

+That's something to find out: more time wasted. If the code says which +version it is we at least know what to run diff against, but that +is not always the case. +

+ + + + +Determining versions + + +

+If a bundled dependency doesn't tell its version we may have to find out +ourselves. Mailing upstream could work, comparing against a number of +tarball contents may work too. Lots of opportunities to waste time. +

+ + + + + +Patching + + +

+Once it is clear that a bundled dependency can be ripped out, a patch is +written, applied and tested (more waste of time). If upstream is willing to +co-operate the patch may be dropped later. If not the patch will need +porting to each new version downstream. +

+ + + + +What to do upstream + + +
    +
  • +

    + Remove bundled dependency: +

    +

    + At best, remove the bundle dependency and allow compilation against + dependency D from either a system-wide installation of it or a + local one at any user-defined location. +

    +

    + That gives flexibility to users on systems without D packaged and makes + it easy to compile against the system copy downstream: cool! +

    +
    • +
  • +

    + Keep bundled dependency: make usage completely optional: +

    +

    + With a build time option to disable use of the bundled dependency it is + possible to bypass it downstream without patching: nice! +

    +

    + When keeping dependency D bundled make sure to follow the upstream of + D closely and update your copy to a recent version of D on every + minor (and major) release to at least reduce the damage done to people + using your bundled version a little. +

    +

    + Also: Clearly document if a bundled dependency is a fork or an unmodified + copy and which version of the bundled software we are dealing with. +

    +
    • +
+ + + +
+ + diff --git a/general-concepts/text.xml b/general-concepts/text.xml index d2b006a5..b397a362 100644 --- a/general-concepts/text.xml +++ b/general-concepts/text.xml @@ -21,6 +21,7 @@ writing ebuilds or working with the Gentoo repository. + From 1793c9e48dc0008d0dc7da6eb642bc32fe4a0da4 Mon Sep 17 00:00:00 2001 From: Sam James Date: Sat, 28 Feb 2026 20:49:44 +0000 Subject: [PATCH 2/4] general-concepts/bundled-deps: 3rd-person perspective; grammar tweaks * Use 3rd-person perspective as we do elsewhere * Tweak grammar (usually just a missing comma or so) * Use longer example names where it aids readability (rather than letters; kept in some places) Signed-off-by: Sam James --- general-concepts/bundled-deps/text.xml | 109 +++++++++++++------------ 1 file changed, 56 insertions(+), 53 deletions(-) diff --git a/general-concepts/bundled-deps/text.xml b/general-concepts/bundled-deps/text.xml index 2dfaf55f..f8380eac 100644 --- a/general-concepts/bundled-deps/text.xml +++ b/general-concepts/bundled-deps/text.xml @@ -16,8 +16,8 @@ explaining the same thing repeatedly by e-mail.

-Say you develop and distribute a piece of software: a game, a library, anything. -Now, the code is considered bundled if any of the following conditions occur: +Code is considered bundled in a piece of software if any of the following +conditions occur:

    @@ -47,7 +47,8 @@ up containing code that does not belong to it.

    There are reasons why bundling dependencies and using static linking occurs; -there are certain benefits to it. So why is it tempting to do such a thing? +there are certain benefits to it. To counter bundling, it is important to +understand why it is appealing to some upstream projects.

    @@ -64,8 +65,8 @@ Windows anyway.

    -It is tempting when using bundled code on Windows to bundle on GNU/Linux too. -It feels consistent and fits together nicely in the mind of the software +It is tempting when using bundled code on Windows to bundle on GNU/Linux +too: it feels consistent and fits together nicely in the mind of the software author.

    @@ -77,17 +78,18 @@ author.

    -If a software package P has some dependency D that is not yet -packaged for major distributions, D makes it harder for P to -get in as packaging P forces the new maintainer to package D -him/herself or to wait for someone else to package it for him/her. +If a software package foomatic has some dependency libbar +that is not yet packaged for major distributions, libbar makes it +harder for foomatic to be packaged, because foomatic forces +the new maintainer to package libbar him/herself or to wait for +someone else to package it for them.

    -Bundling D hides the dependency on D in a way: if the packager -is not paying close attention P may even get in despite and with the -bundled dependency. (It is, however, only a matter of time until someone -notices the bundling.) +Bundling libbar hides the dependency on libbar in a way: +if the packager is not paying close attention foomatic may even get +in despite and with the bundled dependency. (It is, however, only a matter +of time until someone notices the bundling.)

    @@ -98,18 +100,18 @@ notices the bundling.)

    -If P uses a library D, the developers of P may wish -to make some changes to D, for example to add a new feature, modify -the API, or change the default behavior. If the developers of D -for whatever reason are opposed to these changes, the developers of -P may want to fork D. +If foomatic uses a library libbar, the developers of +foomatic may wish to make some changes to libbar, for example +to add a new feature, modify the API, or change the default behavior. +If the developers of libbar for whatever reason are opposed to these +changes, the developers of foomatic may want to fork libbar.

    But publishing and properly maintaining a fork takes time and effort, so -the developers of P could be tempted to take the easy road, bundle -their patched version of D with P, and maybe occasionally -update it for upstream D changes. +the developers of foomatic could be tempted to take the easy road, +bundle their patched version of libbar with foomatic, and +maybe occasionally update it for upstream libbar changes.

    @@ -120,7 +122,7 @@ update it for upstream D changes.

    -So why is bundling dependencies and static linking bad after all? +Why is bundling dependencies and static linking bad after all?

    @@ -129,7 +131,7 @@ So why is bundling dependencies and static linking bad after all?

    -Let's consider you're a developer of foo and your foo uses +Consider the perspective of a baz maintainer where baz uses libbar.

    @@ -144,7 +146,7 @@ minimum.

    If a particular distribution has an efficient security upgrade system, the patched library can get there in less than 24 hours. But that would be of -no use to foo users which will still use the earlier vulnerable library. +no use to baz users which will still use the earlier vulnerable library.

    @@ -153,14 +155,14 @@ Now, depending on how bad things are:

    • - If foo statically linked against libbar, then the users would - either have to rebuild foo themselves to make it use the fixed library - or distribution developers would have to make a new package for foo and + If baz statically linked against libbar, then the users would + either have to rebuild baz themselves to make it use the fixed library + or distribution developers would have to make a new package for baz and make sure it gets to user systems along with libbar (assuming they are aware that the package is statically linked)
    • - If foo bundled a local copy of libbar, then they would have to wait + If baz bundled a local copy of libbar, then they would have to wait till you discover the vulnerability, update libbar sources, release the new version and distributions package the new version
      • @@ -194,8 +196,8 @@ Examples:

      -Say a media player is bundling library libvorbis. If libvorbis is also -installed system-wide this means that two copies of libvorbis: +Say a media player is bundling library libvorbis. If libvorbis +is also installed system-wide, this means that two copies of libvorbis:

        @@ -227,13 +229,13 @@ time are wasted that could have been put to more useful work.

        -If a program P uses a system-installed library A and also uses +If a program foomatic uses a system-installed library A and also uses another library B which bundles library A, there is a potential for symbol collisions.

        -This means that P might use an interface, such as my_function() +This means that foomatic might use an interface, such as my_function() and that the my_function() symbol would be present in both A and the version of A bundled inside of library B.

        @@ -246,8 +248,8 @@ in each copy of A.

        -Since the program P was compiled against the system-installed copy of -A and for various other reasons, if P ends up using the +Since the program foomatic was compiled against the system-installed copy of +A and for various other reasons, if foomatic ends up using the my_function() interface from the version of A bundled in library B instead of the interface in the system-installed copy.

        @@ -293,7 +295,7 @@ bad consequences.

        -So there is a copy of libvorbis bundled with that media player. Which +Suppose there is a copy of libvorbis bundled with a media player. Which version is it? Has it been modified?

        @@ -304,11 +306,11 @@ version is it? Has it been modified?

        Before the bundled dependency can be replaced by the system-widely installed -one, we need to know if it has been modified: we have to know if it's a fork. +one, one must know if it has been modified: is it a fork?

        -If it is a fork it may or may not be replaced without breaking something. +If it is a fork, it may or may not be replaced without breaking something.

        @@ -324,8 +326,8 @@ is not always the case.

        -If a bundled dependency doesn't tell its version we may have to find out -ourselves. Mailing upstream could work, comparing against a number of +If a bundled dependency doesn't share its version, one has to find it +somehow. Mailing upstream could work, comparing against a number of tarball contents may work too. Lots of opportunities to waste time.

        @@ -338,8 +340,8 @@ tarball contents may work too. Lots of opportunities to waste time.

        Once it is clear that a bundled dependency can be ripped out, a patch is -written, applied and tested (more waste of time). If upstream is willing to -co-operate the patch may be dropped later. If not the patch will need +written, applied, and tested (more waste of time). If upstream is willing +to co-operate, the patch may be dropped later. If not, the patch will need porting to each new version downstream.

        @@ -356,30 +358,31 @@ porting to each new version downstream.

        At best, remove the bundle dependency and allow compilation against - dependency D from either a system-wide installation of it or a - local one at any user-defined location. + dependency libbar from either a system-wide installation of it + or a local one at any user-defined location.

        - That gives flexibility to users on systems without D packaged and makes - it easy to compile against the system copy downstream: cool! + That gives flexibility to users on systems without libbar + packaged and makes it easy to compile against the system copy downstream: + cool!

      1. - Keep bundled dependency: make usage completely optional: + Keep bundled dependency: make its use completely optional:

        - With a build time option to disable use of the bundled dependency it is - possible to bypass it downstream without patching: nice! + With a build time option to disable use of the bundled dependency, it + is possible to bypass it downstream without patching: nice!

        - When keeping dependency D bundled make sure to follow the upstream of - D closely and update your copy to a recent version of D on every - minor (and major) release to at least reduce the damage done to people - using your bundled version a little. + When keeping a dependency libbar bundled, make sure to follow the + upstream of libbar closely and update your copy to a recent + version of libbar on every minor (and major) release to at least + reduce the damage done to people using your bundled version a little.

        - Also: Clearly document if a bundled dependency is a fork or an unmodified + Clearly document if a bundled dependency is a fork or an unmodified copy and which version of the bundled software we are dealing with.

        2. From 1b1423ac868d7e5c293c21046eca66e69d84023a Mon Sep 17 00:00:00 2001 From: Sam James Date: Sat, 4 Apr 2026 13:41:02 +0100 Subject: [PATCH 3/4] general-concepts/bundled-deps: mention "bundled vs vulnerable" A common answer when asked to unbundle is that projects will simply update their bundled copy when a vulnerability is discovered. This has two issues: 1) nobody checks if old versions of packages are vulnerable; 2) MITRE does not issue CVEs for bundled versions of dependencies. It's therefore easy to miss that a copy is problematically stale. Suggested-by: Michael Orlitzky Signed-off-by: Sam James --- general-concepts/bundled-deps/text.xml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/general-concepts/bundled-deps/text.xml b/general-concepts/bundled-deps/text.xml index f8380eac..616349eb 100644 --- a/general-concepts/bundled-deps/text.xml +++ b/general-concepts/bundled-deps/text.xml @@ -188,6 +188,13 @@ Examples: fixed in libgd (upstream)
    + +

    +Note further that in general, nobody investigates whether vulnerabilities +impact old versions of packages, which means a stale bundled copy in a +project may be vulnerable without any attention being paid to it. Further, +MITRE does not issue CVEs for bundled dependencies. +

    From 4998068a7599d30176fa4df696c867c2d5c1aa7f Mon Sep 17 00:00:00 2001 From: Sam James Date: Sat, 4 Apr 2026 13:45:59 +0100 Subject: [PATCH 4/4] general-concepts/bundled-deps: mention testsuites, meson subprojects bug #919100 is a nice example of where Meson subprojects exposed a real problem in a bundled dependency. Bug: https://bugs.gentoo.org/919100 Signed-off-by: Sam James --- general-concepts/bundled-deps/text.xml | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/general-concepts/bundled-deps/text.xml b/general-concepts/bundled-deps/text.xml index 616349eb..70a5103f 100644 --- a/general-concepts/bundled-deps/text.xml +++ b/general-concepts/bundled-deps/text.xml @@ -284,6 +284,28 @@ Examples:
+ + +Possibly broken at runtime + + +

+It is uncommon for bundled dependencies to be wired up in the build +system such that their testsuite is built and executed as part of the parent +package. +

+ +

+Missing testsuites mean that real incompatibilities with newer toolchains, +dependencies, or changes in the consuming application can go unnoticed. +

+ +

+Meson subprojects mitigate this by running the testsuite of any bundled +dependencies ('subprojects') by default. +

+ +
@@ -392,6 +414,10 @@ porting to each new version downstream. Clearly document if a bundled dependency is a fork or an unmodified copy and which version of the bundled software we are dealing with.

+

+ Use a Meson subproject which allows clean fallback from a system + copy to a local bundled one. +