bpo-31046: ensurepip does not honour the value of $(prefix)

When cross-compiling, the local Python interpreter that is used
to run `ensurepip` may not have the same value of `sys.prefix` as
the value of the 'prefix' variable that is set in the Makefile.

With the following values used to install Python locally for
a later copy to the files hierarchy owned by the 'termux'
application on an Android device:

    DESTDIR=/tmp/android
    prefix=/data/data/com.termux/files/usr/local

'make install' causes ensurepip to install pip in
$(DESTDIR)/usr/local instead of the expected $(DESTDIR)/$(prefix)
where is installed the standard library.

The attached patch fixes the problem. The patch was implemented
assuming that pip uses distutils for the installation (note that
setup.py also uses the --prefix option in the Makefile), but I
know nothing about pip so forgive me if the patch is wrong and
please just assume it is just a way to demonstrate the problem.

Fixes: #75229
Fixes: https://bugs.python.org/issue31046
Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
References: #17634
Signed-off-by: Matěj Cepl <mcepl@cepl.eu>

Author: 4 people

Doc/library/ensurepip.rst

@@ -65,7 +65,11 @@ is at least as recent as the one available in ``ensurepip``, pass the
 By default, ``pip`` is installed into the current virtual environment
 (if one is active) or into the system site packages (if there is no
 active virtual environment). The installation location can be controlled
-through two additional command line options:
+through some additional command line options:
+
+.. option:: --prefix <dir>
+
+   Installs ``pip`` using the given directory prefix.
 
 .. option:: --root <dir>
 
@@ -108,7 +112,7 @@ Module API
 
 .. function:: bootstrap(root=None, upgrade=False, user=False, \
                         altinstall=False, default_pip=False, \
-                        verbosity=0)
+                        verbosity=0, prefix=None)
 
    Bootstraps ``pip`` into the current or designated environment.
 
@@ -136,6 +140,12 @@ Module API
    *verbosity* controls the level of output to :data:`sys.stdout` from the
    bootstrapping operation.
 
+   *prefix* specifies the directory prefix to use when installing.
+
+   .. versionadded:: 3.14
+
+      The *prefix* parameter.
+
    .. audit-event:: ensurepip.bootstrap root ensurepip.bootstrap
 
    .. note::

Lib/ensurepip/__init__.py

@@ -109,25 +109,25 @@ def _disable_pip_configuration_settings():
 
 def bootstrap(*, root=None, upgrade=False, user=False,
               altinstall=False, default_pip=False,
-              verbosity=0):
+              verbosity=0, prefix=None):
     """
     Bootstrap pip into the current Python installation (or the given root
-    directory).
+    and directory prefix).
 
     Note that calling this function will alter both sys.path and os.environ.
     """
     # Discard the return value
     _bootstrap(root=root, upgrade=upgrade, user=user,
                altinstall=altinstall, default_pip=default_pip,
-               verbosity=verbosity)
+               verbosity=verbosity, prefix=prefix)
 
 
 def _bootstrap(*, root=None, upgrade=False, user=False,
               altinstall=False, default_pip=False,
-              verbosity=0):
+              verbosity=0, prefix=None):
     """
     Bootstrap pip into the current Python installation (or the given root
-    directory). Returns pip command status code.
+    and directory prefix). Returns pip command status code.
 
     Note that calling this function will alter both sys.path and os.environ.
     """
@@ -140,6 +140,8 @@ def _bootstrap(*, root=None, upgrade=False, user=False,
             "to install pip."
         ) from None
 
+    if root is not None and prefix is not None:
+        raise ValueError("Cannot use 'root' and 'prefix' together")
     if altinstall and default_pip:
         raise ValueError("Cannot use altinstall and default_pip together")
 
@@ -172,6 +174,8 @@ def _bootstrap(*, root=None, upgrade=False, user=False,
         args = ["install", "--no-cache-dir", "--no-index", "--find-links", tmpdir]
         if root:
             args += ["--root", root]
+        if prefix:
+            args += ["--prefix", prefix]
         if upgrade:
             args += ["--upgrade"]
         if user:
@@ -249,6 +253,11 @@ def _main(argv=None):
         default=None,
         help="Install everything relative to this alternate root directory.",
     )
+    parser.add_argument(
+        "--prefix",
+        default=None,
+        help="Install everything using this prefix.",
+    )
     parser.add_argument(
         "--altinstall",
         action="store_true",
@@ -268,6 +277,7 @@ def _main(argv=None):
 
     return _bootstrap(
         root=args.root,
+        prefix=args.prefix,
         upgrade=args.upgrade,
         user=args.user,
         verbosity=args.verbosity,

Lib/test/test_ensurepip.py

@@ -121,6 +121,21 @@ def test_bootstrapping_with_root(self):
             unittest.mock.ANY,
         )
 
+    def test_bootstrapping_with_prefix(self):
+        ensurepip.bootstrap(prefix="/foo/bar/")
+        self.run_pip.assert_called_once_with(
+            [
+                "install", "--no-cache-dir", "--no-index", "--find-links",
+                unittest.mock.ANY, "--prefix", "/foo/bar/", "pip",
+            ],
+            unittest.mock.ANY,
+        )
+
+    def test_root_and_prefix_mutual_exclusive(self):
+        with self.assertRaises(ValueError):
+            ensurepip.bootstrap(root="", prefix="")
+        self.assertFalse(self.run_pip.called)
+
     def test_bootstrapping_with_user(self):
         ensurepip.bootstrap(user=True)
 

Makefile.pre.in

@@ -2443,7 +2443,7 @@ install: @FRAMEWORKINSTALLFIRST@ @INSTALLTARGETS@ @FRAMEWORKINSTALLLAST@
 			install|*) ensurepip="" ;; \
 		esac; \
 		$(RUNSHARED) $(PYTHON_FOR_BUILD) -m ensurepip \
-			$$ensurepip --root=$(DESTDIR)/ ; \
+			$$ensurepip --prefix=$(prefix) ; \
 	fi
 
 .PHONY: altinstall
@@ -2454,7 +2454,7 @@ altinstall: commoninstall
 			install|*) ensurepip="--altinstall" ;; \
 		esac; \
 		$(RUNSHARED) $(PYTHON_FOR_BUILD) -m ensurepip \
-			$$ensurepip --root=$(DESTDIR)/ ; \
+			$$ensurepip --prefix=$(prefix) ; \
 	fi
 
 .PHONY: commoninstall

Misc/NEWS.d/next/Library/2019-12-16-17-50-42.bpo-31046.XA-Qfr.rst

@@ -0,0 +1 @@
+A directory prefix can now be specified when using :mod:`ensurepip`.