32796125867

Committed 25 Aug 2026 01:04AM UTC coverage: 83.781% (+0.9%) from 82.885%

Build # 32796125867

Build Type

push

github

Committed by

web-flow

Commit Message

Add Python 3.15 support, drop pre-3.10 workarounds, update vendorized dependencies [release] (#948)

Releases 0.13.21.

## Python 3.15 support

Python 3.15 removed the legacy `load_module()` fallback from the import
machinery, so meta path loaders must implement the PEP 451
`create_module()`/`exec_module()` protocol. Both `VendorImporter` and
`CoverageImporter` failed at import time without it:

```
AttributeError: 'VendorImporter' object has no attribute 'exec_module'
```

There is a subtlety beyond simply adding the two methods.
`module_from_spec()` unconditionally overwrites `__spec__` on whatever
`create_module()` returns, so returning the vendored module directly
replaces its real spec with the alias spec — which has no origin and a
loader unable to locate resources. That surfaced as a second failure:

```
FileNotFoundError: Can't open orphan path (importlib/resources/_adapters.py:139)
```

when virtualenv looked up its own embedded payload. Both loaders now
stash the real spec in `create_module()` and restore it in
`exec_module()`, preserving the pre-3.15 behaviour exactly.

New regression tests (`extern_tests.py`, `coverage_shim_tests.py`) cover
the alias import, submodule import, spec preservation and resource
lookup. They fail without the fix on 3.10, 3.14 and 3.15.

## Pre-3.10 workaround removal

* `collections.abc` fallback in `utils`
* `types.WrapperDescriptorType` et al. fallback in `remote`
* `multiprocessing.semaphore_tracker` fallback in `remote`
* `importlib_metadata` backport fallback in `pip_common`
* 3.8-only `find_distributions` wrapper in `extern`
* hand-rolled `NodeVisitor` computing `end_lineno` in `vendorize`
* `pybuilder_venv.version >= (3, 3)` gate in `sphinx`
* `py2`/`py3` branching in the scaffolded `setup.py`
* `from __future__ import unicode_literals` in `pep517`
* Python 2 residue in `python_utils`
* version guards in the unit and integration tests

The `datetime.UTC` and `unittest.case._addError` fallbacks are retaine... (continued)

Coverage Stats

1427 of 1868 branches covered (76.39%)

Branch coverage included in aggregate %.

27 of 27 new or added lines in 8 files covered. (100.0%)

23 existing lines in 1 file now uncovered.

5619 of 6542 relevant lines covered (85.89%)

38.45 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

57.97

/src/main/python/pybuilder/python_utils.py

#   -*- coding: utf-8 -*-
#
#   This file is part of PyBuilder
#
#   Copyright 2011-2020 PyBuilder Team
#
#   Licensed under the Apache License, Version 2.0 (the "License");
#   you may not use this file except in compliance with the License.
#   You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#   Unless required by applicable law or agreed to in writing, software
#   distributed under the License is distributed on an "AS IS" BASIS,
#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#   See the License for the specific language governing permissions and
#   limitations under the License.

import os
import platform
import sys
import traceback
from collections import OrderedDict
from glob import glob, iglob, escape
from io import StringIO
from os import symlink
from shutil import which


def is_windows(platform=sys.platform, win_platforms={"win32", "cygwin", "msys"}):
    return platform in win_platforms


IS_PYPY = '__pypy__' in sys.builtin_module_names
IS_WIN = is_windows()


def raise_exception(ex, tb):
    raise ex.with_traceback(tb)


def is_string(val):
    return isinstance(val, str)


makedirs = os.makedirs

odict = OrderedDict

_mp_billiard_pyb_env = None  # This will be patched at runtime

_installed_tblib = False

from multiprocessing import log_to_stderr as mp_log_to_stderr, get_context as _mp_get_context  # noqa: E402
from multiprocessing.reduction import ForkingPickler as mp_ForkingPickler  # noqa: E402


def patch_mp_pyb_env(pyb_env):
    global _mp_billiard_pyb_env

    if not _mp_billiard_pyb_env:
        _mp_billiard_pyb_env = pyb_env


def install_tblib():
    global _installed_tblib

    if not _installed_tblib:
        from pybuilder._vendor.tblib import pickling_support

        pickling_support.install()
        _installed_tblib = True


def patch_mp():
    install_tblib()


def mp_get_context(context):
    global _mp_get_context
    return _mp_get_context(context)


def _instrumented_target(q, target, *args, **kwargs):
    patch_mp()

    ex = tb = None
    try:
        send_value = (target(*args, **kwargs), None, None)
    except Exception:
        _, ex, tb = sys.exc_info()
        send_value = (None, ex, tb)

    try:
        q.put(send_value)
    except Exception:
        _, send_ex, send_tb = sys.exc_info()
        e_out = Exception(str(send_ex), send_tb, None if ex is None else str(ex), tb)
        q.put(e_out)


def spawn_process(target=None, args=(), kwargs={}, group=None, name=None):
    """
    Forks a child, making sure that all exceptions from the child are safely sent to the parent
    If a target raises an exception, the exception is re-raised in the parent process
    @return tuple consisting of process exit code and target's return value
    """
    ctx = mp_get_context("spawn")

    q = ctx.SimpleQueue()
    p = ctx.Process(group=group, target=_instrumented_target, name=name, args=[q, target] + list(args), kwargs=kwargs)
    p.start()
    result = q.get()
    p.join()
    if isinstance(result, tuple):
        if result[1]:
            raise_exception(result[1], result[2])
        return p.exitcode, result[0]
    else:
        msg = "Fatal error occurred in the forked process %s: %s" % (p, result.args[0])
        if result.args[2]:
            chained_message = "This error masked the send error '%s':\n%s" % (
                result.args[2], "".join(traceback.format_tb(result.args[3])))
            msg += "\n" + chained_message
        ex = Exception(msg)
        raise_exception(ex, result.args[1])


def prepend_env_to_path(python_env, sys_path):
    """type: (PythonEnv, List(str)) -> None
    Prepend venv directories to sys.path-like collection
    """
    for path in reversed(python_env.site_paths):
        if path not in sys_path:
            sys_path.insert(0, path)


def add_env_to_path(python_env, sys_path):
    """type: (PythonEnv, List(str)) -> None
    Adds venv directories to sys.path-like collection
    """
    for path in python_env.site_paths:
        if path not in sys_path:
            sys_path.append(path)


sys_executable_suffix = sys.executable[len(sys.exec_prefix) + 1:]

python_specific_dir_name = "%s-%s" % (platform.python_implementation().lower(),
                                      ".".join(str(f) for f in sys.version_info))

_, _venv_python_exename = os.path.split(os.path.abspath(getattr(sys, "_base_executable", sys.executable)))

try:
    from imp import load_source
except ImportError:
    from importlib import machinery as importlib_machinery
    from importlib import util as importlib_util
    from importlib._bootstrap import _exec as importlib_exec
    from importlib._bootstrap import _load as importlib_load


    class _HackedGetData:

        """Compatibility support for 'file' arguments of various load_*()
        functions."""

        def __init__(self, fullname, path, file=None):
            super().__init__(fullname, path)
            self.file = file

        def get_data(self, path):
            """Gross hack to contort loader to deal w/ load_*()'s bad API."""
            if self.file and path == self.path:
                # The contract of get_data() requires us to return bytes. Reopen the
                # file in binary mode if needed.
                if not self.file.closed:
                    file = self.file
                    if 'b' not in file.mode:
                        file.close()
                if self.file.closed:
                    self.file = file = open(self.path, 'rb')

                with file:
                    return file.read()
            else:
                return super().get_data(path)


    class _LoadSourceCompatibility(_HackedGetData, importlib_machinery.SourceFileLoader):

        """Compatibility support for implementing load_source()."""


    def load_source(name, pathname, file=None):
        loader = _LoadSourceCompatibility(name, pathname, file)
        spec = importlib_util.spec_from_file_location(name, pathname, loader=loader)
        if name in sys.modules:
            module = importlib_exec(spec, sys.modules[name])
        else:
            module = importlib_load(spec)
        # To allow reloading to potentially work, use a non-hacked loader which
        # won't rely on a now-closed file object.
        module.__loader__ = importlib_machinery.SourceFileLoader(name, pathname)
        module.__spec__.loader = module.__loader__
        return module

__all__ = ["glob", "iglob", "escape"]

1	# -- coding: utf-8 --
2	#
3	# This file is part of PyBuilder
4	#
5	# Copyright 2011-2020 PyBuilder Team
6	#
7	# Licensed under the Apache License, Version 2.0 (the "License");
8	# you may not use this file except in compliance with the License.
9	# You may obtain a copy of the License at
10	#
11	# http://www.apache.org/licenses/LICENSE-2.0
12	#
13	# Unless required by applicable law or agreed to in writing, software
14	# distributed under the License is distributed on an "AS IS" BASIS,
15	# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16	# See the License for the specific language governing permissions and
17	# limitations under the License.
18
19	import os	46✔
20	import platform	46✔
21	import sys	46✔
22	import traceback	46✔
23	from collections import OrderedDict	46✔
24	from glob import glob, iglob, escape	46✔
25	from io import StringIO	46✔
26	from os import symlink	46✔
27	from shutil import which	46✔
28
29
30	def is_windows(platform=sys.platform, win_platforms={"win32", "cygwin", "msys"}):	46✔
31	return platform in win_platforms	46✔
32
33
34	IS_PYPY = '__pypy__' in sys.builtin_module_names	46✔
35	IS_WIN = is_windows()	46✔
36
37
38	def raise_exception(ex, tb):	46✔
39	raise ex.with_traceback(tb)	46✔
40
41
42	def is_string(val):	46✔
43	return isinstance(val, str)	46✔
44
45
46	makedirs = os.makedirs	46✔
47
48	odict = OrderedDict	46✔
49
50	_mp_billiard_pyb_env = None # This will be patched at runtime	46✔
51
52	_installed_tblib = False	46✔
53
54	from multiprocessing import log_to_stderr as mp_log_to_stderr, get_context as _mp_get_context # noqa: E402	46✔
55	from multiprocessing.reduction import ForkingPickler as mp_ForkingPickler # noqa: E402	46✔
56
57
58	def patch_mp_pyb_env(pyb_env):	46✔
59	global _mp_billiard_pyb_env
60
61	if not _mp_billiard_pyb_env:	46✔
62	_mp_billiard_pyb_env = pyb_env	46✔
63
64
65	def install_tblib():	46✔
66	global _installed_tblib
67
68	if not _installed_tblib:	46!
69	from pybuilder._vendor.tblib import pickling_support	46✔
70
71	pickling_support.install()	46✔
72	_installed_tblib = True	46✔
73
74
75	def patch_mp():	46✔
76	install_tblib()	46✔
77
78
79	def mp_get_context(context):	46✔
80	global _mp_get_context
81	return _mp_get_context(context)	46✔
82
83
84	def _instrumented_target(q, target, args, *kwargs):	46✔
UNCOV 85	patch_mp()	×
86
87	ex = tb = None	×
88	try:	×
89	send_value = (target(args, *kwargs), None, None)	×
90	except Exception:	×
91	_, ex, tb = sys.exc_info()	×
92	send_value = (None, ex, tb)	×
93
94	try:	×
95	q.put(send_value)	×
96	except Exception:	×
97	_, send_ex, send_tb = sys.exc_info()	×
98	e_out = Exception(str(send_ex), send_tb, None if ex is None else str(ex), tb)	×
99	q.put(e_out)	×
100
101
102	def spawn_process(target=None, args=(), kwargs={}, group=None, name=None):	46✔
103	"""
104	Forks a child, making sure that all exceptions from the child are safely sent to the parent
105	If a target raises an exception, the exception is re-raised in the parent process
106	@return tuple consisting of process exit code and target's return value
107	"""
108	ctx = mp_get_context("spawn")	×
109
110	q = ctx.SimpleQueue()	×
111	p = ctx.Process(group=group, target=_instrumented_target, name=name, args=[q, target] + list(args), kwargs=kwargs)	×
112	p.start()	×
113	result = q.get()	×
114	p.join()	×
115	if isinstance(result, tuple):	×
116	if result[1]:	×
117	raise_exception(result[1], result[2])	×
118	return p.exitcode, result[0]	×
119	else:
120	msg = "Fatal error occurred in the forked process %s: %s" % (p, result.args[0])	×
121	if result.args[2]:	×
122	chained_message = "This error masked the send error '%s':\n%s" % (	×
123	result.args[2], "".join(traceback.format_tb(result.args[3])))
124	msg += "\n" + chained_message	×
125	ex = Exception(msg)	×
126	raise_exception(ex, result.args[1])	×
127
128
129	def prepend_env_to_path(python_env, sys_path):	46✔
130	"""type: (PythonEnv, List(str)) -> None
131	Prepend venv directories to sys.path-like collection
132	"""
133	for path in reversed(python_env.site_paths):	46✔
134	if path not in sys_path:	46✔
135	sys_path.insert(0, path)	46✔
136
137
138	def add_env_to_path(python_env, sys_path):	46✔
139	"""type: (PythonEnv, List(str)) -> None
140	Adds venv directories to sys.path-like collection
141	"""
142	for path in python_env.site_paths:	×
143	if path not in sys_path:	×
144	sys_path.append(path)	×
145
146
147	sys_executable_suffix = sys.executable[len(sys.exec_prefix) + 1:]	46✔
148
149	python_specific_dir_name = "%s-%s" % (platform.python_implementation().lower(),	46✔
150	".".join(str(f) for f in sys.version_info))
151
152	_, _venv_python_exename = os.path.split(os.path.abspath(getattr(sys, "_base_executable", sys.executable)))	46✔
153
154	try:	46✔
155	from imp import load_source	46✔
UNCOV 156	except ImportError:	30✔
UNCOV 157	from importlib import machinery as importlib_machinery	30✔
UNCOV 158	from importlib import util as importlib_util	30✔
UNCOV 159	from importlib._bootstrap import _exec as importlib_exec	30✔
UNCOV 160	from importlib._bootstrap import _load as importlib_load	30✔
161
162
UNCOV 163	class _HackedGetData:	30✔
164
165	"""Compatibility support for 'file' arguments of various load_*()
166	functions."""
167
UNCOV 168	def __init__(self, fullname, path, file=None):	30✔
UNCOV 169	super().__init__(fullname, path)	30✔
UNCOV 170	self.file = file	30✔
171
UNCOV 172	def get_data(self, path):	30✔
173	"""Gross hack to contort loader to deal w/ load_*()'s bad API."""
UNCOV 174	if self.file and path == self.path:	30!
175	# The contract of get_data() requires us to return bytes. Reopen the
176	# file in binary mode if needed.
177	if not self.file.closed:	×
178	file = self.file	×
179	if 'b' not in file.mode:	×
180	file.close()	×
181	if self.file.closed:	×
182	self.file = file = open(self.path, 'rb')	×
183
184	with file:	×
185	return file.read()	×
186	else:
UNCOV 187	return super().get_data(path)	30✔
188
189
UNCOV 190	class _LoadSourceCompatibility(_HackedGetData, importlib_machinery.SourceFileLoader):	30✔
191
192	"""Compatibility support for implementing load_source()."""
193
194
UNCOV 195	def load_source(name, pathname, file=None):	30✔
UNCOV 196	loader = _LoadSourceCompatibility(name, pathname, file)	30✔
UNCOV 197	spec = importlib_util.spec_from_file_location(name, pathname, loader=loader)	30✔
UNCOV 198	if name in sys.modules:	30✔
UNCOV 199	module = importlib_exec(spec, sys.modules[name])	30✔
200	else:
UNCOV 201	module = importlib_load(spec)	30✔
202	# To allow reloading to potentially work, use a non-hacked loader which
203	# won't rely on a now-closed file object.
UNCOV 204	module.__loader__ = importlib_machinery.SourceFileLoader(name, pathname)	30✔
UNCOV 205	module.__spec__.loader = module.__loader__	30✔
UNCOV 206	return module	30✔
207
208	__all__ = ["glob", "iglob", "escape"]	46✔

pybuilder / pybuilder / 32796125867

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous