19529437518

Committed 20 Nov 2025 07:44AM UTC coverage: 78.884% (-1.4%) from 80.302%

Build # 19529437518

Build Type

push

github

Committed by

web-flow

Commit Message

nfpm.native_libs: Add RPM package depends from packaged pex_binaries (#22899)

## PR Series Overview

This is the second in a series of PRs that introduces a new backend:
`pants.backend.npm.native_libs`
Initially, the backend will be available as:
`pants.backend.experimental.nfpm.native_libs`

I proposed this new backend (originally named `bindeps`) in discussion
#22396.

This backend will inspect ELF bin/lib files (like `lib*.so`) in packaged
contents (for this PR series, only in `pex_binary` targets) to identify
package dependency metadata and inject that metadata on the relevant
`nfpm_deb_package` or `nfpm_rpm_package` targets. Effectively, it will
provide an approximation of these native packager features:
- `rpm`: `rpmdeps` + `elfdeps`
- `deb`: `dh_shlibdeps` + `dpkg-shlibdeps` (These substitute
`${shlibs:Depends}` in debian control files have)

### Goal: Host-agnostic package builds

This pants backend is designed to be host-agnostic, like
[nFPM](https://nfpm.goreleaser.com/).

Native packaging tools are often restricted to a single release of a
single distro. Unlike native package builders, this new pants backend
does not use any of those distro-specific or distro-release-specific
utilities or local package databases. This new backend should be able to
run (help with building deb and rpm packages) anywhere that pants can
run (MacOS, rpm linux distros, deb linux distros, other linux distros,
docker, ...).

### Previous PRs in series

- #22873

## PR Overview

This PR adds rules in `nfpm.native_libs` to add package dependency
metadata to `nfpm_rpm_package`. The 2 new rules are:

- `inject_native_libs_dependencies_in_package_fields`:

    - An implementation of the polymorphic rule `inject_nfpm_package_fields`.
      This rule is low priority (`priority = 2`) so that in-repo plugins can
      override/augment what it injects. (See #22864)

    - Rule logic overview:
        - find any pex_binaries that will be packaged in an `nfpm_rpm_package`
   ... (continued)

Run Details

96 of 118 new or added lines in 3 files covered. (81.36%)

910 existing lines in 53 files now uncovered.

73897 of 93678 relevant lines covered (78.88%)

3.21 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

79.41

/src/python/pants/option/option_value_container.py

# Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).

from __future__ import annotations

import copy
from collections.abc import Iterator
from dataclasses import dataclass

from pants.option.ranked_value import Rank, RankedValue, Value

Key = str


class OptionValueContainerBuilder:
    def __init__(self, value_map: dict[Key, RankedValue] | None = None) -> None:
        self._value_map: dict[Key, RankedValue] = value_map if value_map else {}

    def update(self, other: OptionValueContainerBuilder) -> None:
        """Set other's values onto this object.

        For each key, highest ranked value wins. In a tie, other's value wins.
        """
        for k, v in other._value_map.items():
            self._set(k, v)

    def _set(self, key: Key, value: RankedValue) -> None:
        if not isinstance(value, RankedValue):
            raise AttributeError(f"Value must be of type RankedValue: {value}")

        existing_value = self._value_map.get(key)
        existing_rank = existing_value.rank if existing_value is not None else Rank.NONE
        if value.rank >= existing_rank:
            # We set values from outer scopes before values from inner scopes, so
            # in case of equal rank we overwrite. That way that the inner scope value wins.
            self._value_map[key] = value

    # Support attribute setting, e.g., opts.foo = RankedValue(Rank.HARDCODED, 42).
    def __setattr__(self, key: Key, value: RankedValue) -> None:
        if key == "_value_map":
            return super().__setattr__(key, value)
        self._set(key, value)

    def build(self) -> OptionValueContainer:
        return OptionValueContainer(copy.copy(self._value_map))


@dataclass(frozen=True)
class OptionValueContainer:
    """A container for option values.

    Implements "value ranking":

       Attribute values can be ranked, so that a given attribute's value can only be changed if
       the new value has at least as high a rank as the old value. This allows an option value in
       an outer scope to override that option's value in an inner scope, when the outer scope's
       value comes from a higher ranked source (e.g., the outer value comes from an env var and
       the inner one from config).

       See ranked_value.py for more details.
    """

    _value_map: dict[Key, RankedValue]

    def get_keys(self) -> set[Key]:
        return set(self._value_map.keys())

    def get_explicit_keys(self) -> list[Key]:
        """Returns the keys for any values that were set explicitly (via flag, config, or env
        var)."""
        ret = []
        for k, v in self._value_map.items():
            if v.rank > Rank.CONFIG_DEFAULT:
                ret.append(k)
        return ret

    def get_rank(self, key: Key) -> Rank:
        """Returns the rank of the value at the specified key.

        Returns one of the constants in Rank.
        """
        ranked_value = self._value_map.get(key)
        if ranked_value is None:
            raise AttributeError(key)
        return ranked_value.rank

    def is_flagged(self, key: Key) -> bool:
        """Returns `True` if the value for the specified key was supplied via a flag.

        A convenience equivalent to `get_rank(key) == Rank.FLAG`.

        This check can be useful to determine whether or not a user explicitly set an option for this
        run.  Although a user might also set an option explicitly via an environment variable, ie via:
        `ENV_VAR=value ./pants ...`, this is an ambiguous case since the environment variable could also
        be permanently set in the user's environment.

        :param string key: The name of the option to check.
        :returns: `True` if the option was explicitly flagged by the user from the command line.
        :rtype: bool
        """
        return self.get_rank(key) == Rank.FLAG

    def is_default(self, key: Key) -> bool:
        """Returns `True` if the value for the specified key was not supplied by the user.

        I.e. the option was NOT specified config files, on the cli, or in environment variables.

        :param string key: The name of the option to check.
        :returns: `True` if the user did not set the value for this option.
        :rtype: bool
        """
        return self.get_rank(key) in (Rank.NONE, Rank.HARDCODED)

    def get(self, key: Key, default: Value | None = None):
        # Support dict-like dynamic access.  See also __getitem__ below.
        if key not in self._value_map:
            return default
        return self._get_underlying_value(key)

    def as_dict(self) -> dict[Key, Value]:
        return {key: self.get(key) for key in self._value_map}

    def to_builder(self) -> OptionValueContainerBuilder:
        return OptionValueContainerBuilder(copy.copy(self._value_map))

    def _get_underlying_value(self, key: Key):
        # Note that the key may exist with a value of None, so we can't just
        # test self._value_map.get() for None.
        if key not in self._value_map:
            raise AttributeError(key)
        ranked_val = self._value_map[key]
        return ranked_val.value

    # Support natural dynamic access, e.g., opts[foo] is more idiomatic than getattr(opts, 'foo').
    def __getitem__(self, key: Key):
        return self.__getattr__(key)

    # Support attribute getting, e.g., foo = opts.foo.
    # Note: Called only if regular attribute lookup fails,
    # so method and member access will be handled the normal way.
    def __getattr__(self, key: Key):
        if key == "_value_map":
            # In case we get called in copy, which doesn't invoke the ctor.
            raise AttributeError(key)
        return self._get_underlying_value(key)

    def __contains__(self, key: Key):
        return key in self._value_map

    def __iter__(self) -> Iterator[Key]:
        """Returns an iterator over all option names, in lexicographical order."""
        yield from sorted(self._value_map.keys())

1	# Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).
2	# Licensed under the Apache License, Version 2.0 (see LICENSE).
3
4	from __future__ import annotations	11✔
5
6	import copy	11✔
7	from collections.abc import Iterator	11✔
8	from dataclasses import dataclass	11✔
9
10	from pants.option.ranked_value import Rank, RankedValue, Value	11✔
11
12	Key = str	11✔
13
14
15	class OptionValueContainerBuilder:	11✔
16	def __init__(self, value_map: dict[Key, RankedValue] \| None = None) -> None:	11✔
17	self._value_map: dict[Key, RankedValue] = value_map if value_map else {}	11✔
18
19	def update(self, other: OptionValueContainerBuilder) -> None:	11✔
20	"""Set other's values onto this object.
21
22	For each key, highest ranked value wins. In a tie, other's value wins.
23	"""
UNCOV 24	for k, v in other._value_map.items():	×
UNCOV 25	self._set(k, v)	×
26
27	def _set(self, key: Key, value: RankedValue) -> None:	11✔
28	if not isinstance(value, RankedValue):	11✔
29	raise AttributeError(f"Value must be of type RankedValue: {value}")	×
30
31	existing_value = self._value_map.get(key)	11✔
32	existing_rank = existing_value.rank if existing_value is not None else Rank.NONE	11✔
33	if value.rank >= existing_rank:	11✔
34	# We set values from outer scopes before values from inner scopes, so
35	# in case of equal rank we overwrite. That way that the inner scope value wins.
36	self._value_map[key] = value	11✔
37
38	# Support attribute setting, e.g., opts.foo = RankedValue(Rank.HARDCODED, 42).
39	def __setattr__(self, key: Key, value: RankedValue) -> None:	11✔
40	if key == "_value_map":	11✔
41	return super().__setattr__(key, value)	11✔
42	self._set(key, value)	11✔
43
44	def build(self) -> OptionValueContainer:	11✔
45	return OptionValueContainer(copy.copy(self._value_map))	11✔
46
47
48	@dataclass(frozen=True)	11✔
49	class OptionValueContainer:	11✔
50	"""A container for option values.
51
52	Implements "value ranking":
53
54	Attribute values can be ranked, so that a given attribute's value can only be changed if
55	the new value has at least as high a rank as the old value. This allows an option value in
56	an outer scope to override that option's value in an inner scope, when the outer scope's
57	value comes from a higher ranked source (e.g., the outer value comes from an env var and
58	the inner one from config).
59
60	See ranked_value.py for more details.
61	"""
62
63	_value_map: dict[Key, RankedValue]	11✔
64
65	def get_keys(self) -> set[Key]:	11✔
66	return set(self._value_map.keys())	×
67
68	def get_explicit_keys(self) -> list[Key]:	11✔
69	"""Returns the keys for any values that were set explicitly (via flag, config, or env
70	var)."""
UNCOV 71	ret = []	×
UNCOV 72	for k, v in self._value_map.items():	×
UNCOV 73	if v.rank > Rank.CONFIG_DEFAULT:	×
UNCOV 74	ret.append(k)	×
UNCOV 75	return ret	×
76
77	def get_rank(self, key: Key) -> Rank:	11✔
78	"""Returns the rank of the value at the specified key.
79
80	Returns one of the constants in Rank.
81	"""
82	ranked_value = self._value_map.get(key)	4✔
83	if ranked_value is None:	4✔
84	raise AttributeError(key)	×
85	return ranked_value.rank	4✔
86
87	def is_flagged(self, key: Key) -> bool:	11✔
88	"""Returns `True` if the value for the specified key was supplied via a flag.
89
90	A convenience equivalent to `get_rank(key) == Rank.FLAG`.
91
92	This check can be useful to determine whether or not a user explicitly set an option for this
93	run. Although a user might also set an option explicitly via an environment variable, ie via:
94	`ENV_VAR=value ./pants ...`, this is an ambiguous case since the environment variable could also
95	be permanently set in the user's environment.
96
97	:param string key: The name of the option to check.
98	:returns: `True` if the option was explicitly flagged by the user from the command line.
99	:rtype: bool
100	"""
101	return self.get_rank(key) == Rank.FLAG	2✔
102
103	def is_default(self, key: Key) -> bool:	11✔
104	"""Returns `True` if the value for the specified key was not supplied by the user.
105
106	I.e. the option was NOT specified config files, on the cli, or in environment variables.
107
108	:param string key: The name of the option to check.
109	:returns: `True` if the user did not set the value for this option.
110	:rtype: bool
111	"""
112	return self.get_rank(key) in (Rank.NONE, Rank.HARDCODED)	2✔
113
114	def get(self, key: Key, default: Value \| None = None):	11✔
115	# Support dict-like dynamic access. See also __getitem__ below.
116	if key not in self._value_map:	2✔
117	return default	1✔
118	return self._get_underlying_value(key)	2✔
119
120	def as_dict(self) -> dict[Key, Value]:	11✔
UNCOV 121	return {key: self.get(key) for key in self._value_map}	×
122
123	def to_builder(self) -> OptionValueContainerBuilder:	11✔
124	return OptionValueContainerBuilder(copy.copy(self._value_map))	×
125
126	def _get_underlying_value(self, key: Key):	11✔
127	# Note that the key may exist with a value of None, so we can't just
128	# test self._value_map.get() for None.
129	if key not in self._value_map:	11✔
130	raise AttributeError(key)	1✔
131	ranked_val = self._value_map[key]	11✔
132	return ranked_val.value	11✔
133
134	# Support natural dynamic access, e.g., opts[foo] is more idiomatic than getattr(opts, 'foo').
135	def __getitem__(self, key: Key):	11✔
136	return self.__getattr__(key)	2✔
137
138	# Support attribute getting, e.g., foo = opts.foo.
139	# Note: Called only if regular attribute lookup fails,
140	# so method and member access will be handled the normal way.
141	def __getattr__(self, key: Key):	11✔
142	if key == "_value_map":	11✔
143	# In case we get called in copy, which doesn't invoke the ctor.
144	raise AttributeError(key)	×
145	return self._get_underlying_value(key)	11✔
146
147	def __contains__(self, key: Key):	11✔
148	return key in self._value_map	×
149
150	def __iter__(self) -> Iterator[Key]:	11✔
151	"""Returns an iterator over all option names, in lexicographical order."""
152	yield from sorted(self._value_map.keys())	1✔

pantsbuild / pants / 19529437518

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous