pantsbuild / pants / 29631338772 / 6
93%
main: 93%

DEFAULT BRANCH: main
Ran
Files 1223
Run time 59s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 54.042% (+0.05%) from 53.992%
29631338772.6

push

github

web-flow 
feat(golang): add module level granularity for third party dependencies (#23499)

This PR adds a new opt-in model for go third party module dependencies,
which is one target per module instead of one target per package.

### Background

The problem today is that some dependencies, for example cloud provider
libraries, contain a lot of packages. Currently the go backend maps each
third party package to a generated target, and infers the entire
dependency graph between all packages across all third party modules. In
a repo with 800+ third party dependencies, this leads to **over 12000
third party package targets** that all go through a dependency inference
walk. This leads to a lot of engine overhead. This is especially painful
for CI when running `--changed-since` with `--changed-dependents`, since
every single CI run is hit by this at startup regardless if golang is
affected by the change.

In my case with a multi-language repo where Go is under-represented,
this inference is the majority cost of pants startup on all runs.

This improvement goes beyond just dependency inference, as shown in
benchmarks, but it was the main driver for the change.

### This PR

Adds a new option `[golang].third_party_target_granularity` with the
options `package` or `module`. `package` retains the current behavior of
generating targets per third party package. `module` only creates one
target per third party module.

I've added the two modes as moving to `module` has some theoretical
breaking cases since third party packages are not addressable with this
mode. So any explicit dependency references or a main-reference into a
third party package would no longer work.

I have instead added another field `main_import_path` to `go_binary`
which covers targeting a third party package as the main package even
when using `module` granularity. This field is also supported for
`package` mode.

### Implementation notes

Module targets are represented by the existing `go_third_party... (continued)

34621 of 64063 relevant lines covered (54.04%)

0.54 hits per line

Source Files on job test_python_linux_x86_64_9/10 - 29631338772.6