19529437518

Committed 20 Nov 2025 07:44AM UTC coverage: 78.884% (-1.4%) from 80.302%

Build # 19529437518

Build Type

push

github

Committed by

web-flow

Commit Message

nfpm.native_libs: Add RPM package depends from packaged pex_binaries (#22899)

## PR Series Overview

This is the second in a series of PRs that introduces a new backend:
`pants.backend.npm.native_libs`
Initially, the backend will be available as:
`pants.backend.experimental.nfpm.native_libs`

I proposed this new backend (originally named `bindeps`) in discussion
#22396.

This backend will inspect ELF bin/lib files (like `lib*.so`) in packaged
contents (for this PR series, only in `pex_binary` targets) to identify
package dependency metadata and inject that metadata on the relevant
`nfpm_deb_package` or `nfpm_rpm_package` targets. Effectively, it will
provide an approximation of these native packager features:
- `rpm`: `rpmdeps` + `elfdeps`
- `deb`: `dh_shlibdeps` + `dpkg-shlibdeps` (These substitute
`${shlibs:Depends}` in debian control files have)

### Goal: Host-agnostic package builds

This pants backend is designed to be host-agnostic, like
[nFPM](https://nfpm.goreleaser.com/).

Native packaging tools are often restricted to a single release of a
single distro. Unlike native package builders, this new pants backend
does not use any of those distro-specific or distro-release-specific
utilities or local package databases. This new backend should be able to
run (help with building deb and rpm packages) anywhere that pants can
run (MacOS, rpm linux distros, deb linux distros, other linux distros,
docker, ...).

### Previous PRs in series

- #22873

## PR Overview

This PR adds rules in `nfpm.native_libs` to add package dependency
metadata to `nfpm_rpm_package`. The 2 new rules are:

- `inject_native_libs_dependencies_in_package_fields`:

    - An implementation of the polymorphic rule `inject_nfpm_package_fields`.
      This rule is low priority (`priority = 2`) so that in-repo plugins can
      override/augment what it injects. (See #22864)

    - Rule logic overview:
        - find any pex_binaries that will be packaged in an `nfpm_rpm_package`
   ... (continued)

Run Details

96 of 118 new or added lines in 3 files covered. (81.36%)

910 existing lines in 53 files now uncovered.

73897 of 93678 relevant lines covered (78.88%)

3.21 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

74.39

/src/python/pants/bsp/spec/base.py

# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
# Licensed under the Apache License, Version 2.0 (see LICENSE).
from __future__ import annotations

from dataclasses import dataclass
from enum import IntEnum
from typing import Any, ClassVar

from pants.build_graph.address import Address

# -----------------------------------------------------------------------------------------------
# Basic JSON Structures
# See https://build-server-protocol.github.io/docs/specification.html#basic-json-structures
# -----------------------------------------------------------------------------------------------

Uri = str


@dataclass(frozen=True)
class BuildTargetIdentifier:
    """A unique identifier for a target, can use any URI-compatible encoding as long as it is unique
    within the workspace.

    Clients should not infer metadata out of the URI structure such as the path or query parameters,
    use BuildTarget instead.
    """

    # The target’s Uri
    uri: Uri

    @classmethod
    def from_json_dict(cls, d):
        return cls(uri=d["uri"])

    def to_json_dict(self):
        return {"uri": self.uri}

    @classmethod
    def from_address(cls, addr: Address) -> BuildTargetIdentifier:
        return cls(uri=f"pants:{str(addr)}")


@dataclass(frozen=True)
class BuildTargetCapabilities:
    # This target can be compiled by the BSP server.
    can_compile: bool = False

    # This target can be tested by the BSP server.
    can_test: bool = False

    # This target can be run by the BSP server.
    can_run: bool = False

    # This target can be debugged by the BSP server.
    can_debug: bool = False

    @classmethod
    def from_json_dict(cls, d):
        return cls(
            can_compile=d["canCompile"],
            can_test=d["canTest"],
            can_run=d["canRun"],
            can_debug=d["canDebug"],
        )

    def to_json_dict(self):
        return {
            "canCompile": self.can_compile,
            "canTest": self.can_test,
            "canRun": self.can_run,
            "canDebug": self.can_debug,
        }


# Note: The BSP "build target" concept is _not_ the same as a Pants "target". They are similar but
# should be not be conflated with one another.
@dataclass(frozen=True)
class BuildTarget:
    """Build target contains metadata about an artifact (for example library, test, or binary
    artifact)"""

    # The target’s unique identifier
    id: BuildTargetIdentifier

    # A human-readable name for this target.
    # May be presented in the user interface.
    # Should be unique if possible.
    # The id.uri is used if None.
    display_name: str | None

    # The directory where this target belongs to. Multiple build targets are allowed to map
    # to the same base directory, and a build target is not required to have a base directory.
    # A base directory does not determine the sources of a target, see buildTarget/sources. */
    base_directory: Uri | None

    # Free-form string tags to categorize or label this build target.
    # For example, can be used by the client to:
    # - customize how the target should be translated into the client's project model.
    # - group together different but related targets in the user interface.
    # - display icons or colors in the user interface.
    # Pre-defined tags are listed in `BuildTargetTag` but clients and servers
    # are free to define new tags for custom purposes.
    tags: tuple[str, ...]

    # The capabilities of this build target.
    capabilities: BuildTargetCapabilities

    # The set of languages that this target contains.
    # The ID string for each language is defined in the LSP.
    language_ids: tuple[str, ...]

    # The direct upstream build target dependencies of this build target
    dependencies: tuple[BuildTargetIdentifier, ...]

    # Language-specific metadata about this target.
    # See ScalaBuildTarget as an example.
    data: BSPData | None

    @classmethod
    def from_json_dict(cls, d):
        return cls(
            id=BuildTargetIdentifier.from_json_dict(d["id"]),
            display_name=d.get("displayName"),
            base_directory=d["baseDirectory"],
            tags=tuple(d.get("tags", [])),
            capabilities=BuildTargetCapabilities.from_json_dict(d["capabilities"]),
            language_ids=tuple(d.get("languageIds", [])),
            dependencies=tuple(
                BuildTargetIdentifier.from_json_dict(x) for x in d.get("dependencies", [])
            ),
            # data_kind=d.get("dataKind"),  # TODO: figure out generic decode, this is only used in tests!
            data=d.get("data"),
        )

    def to_json_dict(self):
        result = {
            "id": self.id.to_json_dict(),
            "capabilities": self.capabilities.to_json_dict(),
            "tags": self.tags,
            "languageIds": self.language_ids,
            "dependencies": [dep.to_json_dict() for dep in self.dependencies],
        }
        if self.display_name is not None:
            result["displayName"] = self.display_name
        if self.base_directory is not None:
            result["baseDirectory"] = self.base_directory
        if self.data is not None:
            result["dataKind"] = self.data.DATA_KIND
            result["data"] = self.data.to_json_dict()
        return result


class BuildTargetDataKind:
    # The `data` field contains a `ScalaBuildTarget` object.
    SCALA = "scala"

    # The `data` field contains a `SbtBuildTarget` object.
    SBT = "sbt"


class BuildTargetTag:
    # Target contains re-usable functionality for downstream targets. May have any
    # combination of capabilities.
    LIBRARY = "library"

    # Target contains source code for producing any kind of application, may have
    # but does not require the `canRun` capability.
    APPLICATION = "application"

    # Target contains source code for testing purposes, may have but does not
    # require the `canTest` capability.
    TEST = "test"

    # Target contains source code for integration testing purposes, may have
    # but does not require the `canTest` capability.
    # The difference between "test" and "integration-test" is that
    # integration tests traditionally run slower compared to normal tests
    # and require more computing resources to execute.
    INTEGRATION_TEST = "integration-test"

    # Target contains source code to measure performance of a program, may have
    # but does not require the `canRun` build target capability.
    BENCHMARK = "benchmark"

    # Target should be ignored by IDEs. */
    NO_IDE = "no-ide"

    # Actions on the target such as build and test should only be invoked manually
    # and explicitly. For example, triggering a build on all targets in the workspace
    # should by default not include this target.
    #
    # The original motivation to add the "manual" tag comes from a similar functionality
    # that exists in Bazel, where targets with this tag have to be specified explicitly
    # on the command line.
    MANUAL = "manual"


@dataclass(frozen=True)
class TaskId:
    """The Task Id allows clients to uniquely identify a BSP task and establish a client-parent
    relationship with another task id."""

    # A unique identifier
    id: str

    # The parent task ids, if any. A non-empty parents field means
    # this task is a sub-task of every parent task id. The child-parent
    # relationship of tasks makes it possible to render tasks in
    # a tree-like user interface or inspect what caused a certain task
    # execution.
    parents: tuple[str, ...] | None = None

    @classmethod
    def from_json_dict(cls, d):
        return cls(
            id=d["id"],
            parents=tuple(d["parents"]) if "parents" in d else None,
        )

    def to_json_dict(self) -> dict[str, Any]:
        result: dict[str, Any] = {
            "id": self.id,
        }
        if self.parents is not None:
            result["parents"] = self.parents
        return result


class StatusCode(IntEnum):
    # Execution was successful.
    OK = 1

    # Execution failed.
    ERROR = 2

    # Execution was cancelled.
    CANCELLED = 3


class BSPData:
    """Mix-in for BSP spec types that can live in a data field."""

    DATA_KIND: ClassVar[str]

    def to_json_dict(self) -> dict[str, Any]:
        raise NotImplementedError

1	# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md).
2	# Licensed under the Apache License, Version 2.0 (see LICENSE).
3	from __future__ import annotations	1✔
4
5	from dataclasses import dataclass	1✔
6	from enum import IntEnum	1✔
7	from typing import Any, ClassVar	1✔
8
9	from pants.build_graph.address import Address	1✔
10
11	# -----------------------------------------------------------------------------------------------
12	# Basic JSON Structures
13	# See https://build-server-protocol.github.io/docs/specification.html#basic-json-structures
14	# -----------------------------------------------------------------------------------------------
15
16	Uri = str	1✔
17
18
19	@dataclass(frozen=True)	1✔
20	class BuildTargetIdentifier:	1✔
21	"""A unique identifier for a target, can use any URI-compatible encoding as long as it is unique
22	within the workspace.
23
24	Clients should not infer metadata out of the URI structure such as the path or query parameters,
25	use BuildTarget instead.
26	"""
27
28	# The target’s Uri
29	uri: Uri	1✔
30
31	@classmethod	1✔
32	def from_json_dict(cls, d):	1✔
UNCOV 33	return cls(uri=d["uri"])	×
34
35	def to_json_dict(self):	1✔
UNCOV 36	return {"uri": self.uri}	×
37
38	@classmethod	1✔
39	def from_address(cls, addr: Address) -> BuildTargetIdentifier:	1✔
40	return cls(uri=f"pants:{str(addr)}")	×
41
42
43	@dataclass(frozen=True)	1✔
44	class BuildTargetCapabilities:	1✔
45	# This target can be compiled by the BSP server.
46	can_compile: bool = False	1✔
47
48	# This target can be tested by the BSP server.
49	can_test: bool = False	1✔
50
51	# This target can be run by the BSP server.
52	can_run: bool = False	1✔
53
54	# This target can be debugged by the BSP server.
55	can_debug: bool = False	1✔
56
57	@classmethod	1✔
58	def from_json_dict(cls, d):	1✔
UNCOV 59	return cls(	×
60	can_compile=d["canCompile"],
61	can_test=d["canTest"],
62	can_run=d["canRun"],
63	can_debug=d["canDebug"],
64	)
65
66	def to_json_dict(self):	1✔
UNCOV 67	return {	×
68	"canCompile": self.can_compile,
69	"canTest": self.can_test,
70	"canRun": self.can_run,
71	"canDebug": self.can_debug,
72	}
73
74
75	# Note: The BSP "build target" concept is _not_ the same as a Pants "target". They are similar but
76	# should be not be conflated with one another.
77	@dataclass(frozen=True)	1✔
78	class BuildTarget:	1✔
79	"""Build target contains metadata about an artifact (for example library, test, or binary
80	artifact)"""
81
82	# The target’s unique identifier
83	id: BuildTargetIdentifier	1✔
84
85	# A human-readable name for this target.
86	# May be presented in the user interface.
87	# Should be unique if possible.
88	# The id.uri is used if None.
89	display_name: str \| None	1✔
90
91	# The directory where this target belongs to. Multiple build targets are allowed to map
92	# to the same base directory, and a build target is not required to have a base directory.
93	# A base directory does not determine the sources of a target, see buildTarget/sources. */
94	base_directory: Uri \| None	1✔
95
96	# Free-form string tags to categorize or label this build target.
97	# For example, can be used by the client to:
98	# - customize how the target should be translated into the client's project model.
99	# - group together different but related targets in the user interface.
100	# - display icons or colors in the user interface.
101	# Pre-defined tags are listed in `BuildTargetTag` but clients and servers
102	# are free to define new tags for custom purposes.
103	tags: tuple[str, ...]	1✔
104
105	# The capabilities of this build target.
106	capabilities: BuildTargetCapabilities	1✔
107
108	# The set of languages that this target contains.
109	# The ID string for each language is defined in the LSP.
110	language_ids: tuple[str, ...]	1✔
111
112	# The direct upstream build target dependencies of this build target
113	dependencies: tuple[BuildTargetIdentifier, ...]	1✔
114
115	# Language-specific metadata about this target.
116	# See ScalaBuildTarget as an example.
117	data: BSPData \| None	1✔
118
119	@classmethod	1✔
120	def from_json_dict(cls, d):	1✔
UNCOV 121	return cls(	×
122	id=BuildTargetIdentifier.from_json_dict(d["id"]),
123	display_name=d.get("displayName"),
124	base_directory=d["baseDirectory"],
125	tags=tuple(d.get("tags", [])),
126	capabilities=BuildTargetCapabilities.from_json_dict(d["capabilities"]),
127	language_ids=tuple(d.get("languageIds", [])),
128	dependencies=tuple(
129	BuildTargetIdentifier.from_json_dict(x) for x in d.get("dependencies", [])
130	),
131	# data_kind=d.get("dataKind"), # TODO: figure out generic decode, this is only used in tests!
132	data=d.get("data"),
133	)
134
135	def to_json_dict(self):	1✔
UNCOV 136	result = {	×
137	"id": self.id.to_json_dict(),
138	"capabilities": self.capabilities.to_json_dict(),
139	"tags": self.tags,
140	"languageIds": self.language_ids,
141	"dependencies": [dep.to_json_dict() for dep in self.dependencies],
142	}
UNCOV 143	if self.display_name is not None:	×
UNCOV 144	result["displayName"] = self.display_name	×
UNCOV 145	if self.base_directory is not None:	×
UNCOV 146	result["baseDirectory"] = self.base_directory	×
UNCOV 147	if self.data is not None:	×
UNCOV 148	result["dataKind"] = self.data.DATA_KIND	×
UNCOV 149	result["data"] = self.data.to_json_dict()	×
UNCOV 150	return result	×
151
152
153	class BuildTargetDataKind:	1✔
154	# The `data` field contains a `ScalaBuildTarget` object.
155	SCALA = "scala"	1✔
156
157	# The `data` field contains a `SbtBuildTarget` object.
158	SBT = "sbt"	1✔
159
160
161	class BuildTargetTag:	1✔
162	# Target contains re-usable functionality for downstream targets. May have any
163	# combination of capabilities.
164	LIBRARY = "library"	1✔
165
166	# Target contains source code for producing any kind of application, may have
167	# but does not require the `canRun` capability.
168	APPLICATION = "application"	1✔
169
170	# Target contains source code for testing purposes, may have but does not
171	# require the `canTest` capability.
172	TEST = "test"	1✔
173
174	# Target contains source code for integration testing purposes, may have
175	# but does not require the `canTest` capability.
176	# The difference between "test" and "integration-test" is that
177	# integration tests traditionally run slower compared to normal tests
178	# and require more computing resources to execute.
179	INTEGRATION_TEST = "integration-test"	1✔
180
181	# Target contains source code to measure performance of a program, may have
182	# but does not require the `canRun` build target capability.
183	BENCHMARK = "benchmark"	1✔
184
185	# Target should be ignored by IDEs. */
186	NO_IDE = "no-ide"	1✔
187
188	# Actions on the target such as build and test should only be invoked manually
189	# and explicitly. For example, triggering a build on all targets in the workspace
190	# should by default not include this target.
191	#
192	# The original motivation to add the "manual" tag comes from a similar functionality
193	# that exists in Bazel, where targets with this tag have to be specified explicitly
194	# on the command line.
195	MANUAL = "manual"	1✔
196
197
198	@dataclass(frozen=True)	1✔
199	class TaskId:	1✔
200	"""The Task Id allows clients to uniquely identify a BSP task and establish a client-parent
201	relationship with another task id."""
202
203	# A unique identifier
204	id: str	1✔
205
206	# The parent task ids, if any. A non-empty parents field means
207	# this task is a sub-task of every parent task id. The child-parent
208	# relationship of tasks makes it possible to render tasks in
209	# a tree-like user interface or inspect what caused a certain task
210	# execution.
211	parents: tuple[str, ...] \| None = None	1✔
212
213	@classmethod	1✔
214	def from_json_dict(cls, d):	1✔
215	return cls(	×
216	id=d["id"],
217	parents=tuple(d["parents"]) if "parents" in d else None,
218	)
219
220	def to_json_dict(self) -> dict[str, Any]:	1✔
221	result: dict[str, Any] = {	×
222	"id": self.id,
223	}
224	if self.parents is not None:	×
225	result["parents"] = self.parents	×
226	return result	×
227
228
229	class StatusCode(IntEnum):	1✔
230	# Execution was successful.
231	OK = 1	1✔
232
233	# Execution failed.
234	ERROR = 2	1✔
235
236	# Execution was cancelled.
237	CANCELLED = 3	1✔
238
239
240	class BSPData:	1✔
241	"""Mix-in for BSP spec types that can live in a data field."""
242
243	DATA_KIND: ClassVar[str]	1✔
244
245	def to_json_dict(self) -> dict[str, Any]:	1✔
246	raise NotImplementedError	×

pantsbuild / pants / 19529437518

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous