pantsbuild / pants / 36000
0%
main: 79%

LAST BUILD BRANCH: csb/pants-scie-experimental
DEFAULT BRANCH: main
Ran
Jobs 1
Files 434
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
36000

push

travis-ci-com

web-flow 
Fix bug with linter targets being skipped (#10974) (#10975)

### Problem

We noticed an issue where, when running the `./pants lint` command on a large number of targets in a repository, some targets were being completely skipped by the flake8 process, resulting in the flake8 linter output falsely reporting all good, when there were actually files in the repo with linter errors.

The problem turned out to lie in the `group_field_sets_by_constraints` method. This method takes as its input an unsorted collection of field sets corresponding to the input targets, and groups them by their python interpreter contraint. This method is used as part of the pipeline for running the flake8 process on python source files.

Internally, this method calls the python standard library `itertools.groupby` method. It turns out that `groupby` does not work as expected with unsorted input data - it generates a new sub-iterable every time the sorting key changes (in this case, the interpreter constraint), rather than creating as many sub-iterables as there were distinct sorting keys in the input data. Because we were taking the output of this method and using it in a dictionary comprehension, we were accidentally overwriting dictionary values in a non-deterministic way, resulting in some filed sets getting skipped before the flake8 process could run on them.

### Solution

`group_field_sets_by_constraints` was rewritten to avoid using `itertools.groupby` altogether, so we no longer skip inputs; and a test was added to make sure that we handle unsorted field set inputs to this method correctly.

Co-authored-by: gshuflin <greg.shuflin@toolchain.com>

0 of 29723 relevant lines covered (0.0%)

0.0 hits per line

Jobs
ID Job ID Ran Files Coverage
5 36000.5 (PANTS_CONFIG_FILES="${TRAVIS_BUILD_DIR}/pants.travis-ci.toml" PANTS_DYNAMIC_UI=false LC_ALL="en_US.UTF-8" AWS_BUCKET=ci-public.pantsbuild.org BOOTSTRAPPED_PEX_KEY_PREFIX=daily/${TRAVIS_BUILD_NUMBER}/${TRAVIS_BUILD_ID}/pants.pex NATIVE_ENGINE_SO_KEY_PREFIX=monthly/native_engine_so PYENV_PY27_VERSION=2.7.18 PYENV_PY36_VERSION=3.6.10 PYENV_PY37_VERSION=3.7.7 PYENV_PY38_VERSION=3.8.3 PYENV_ROOT_OSX=${HOME}/.pants_pyenv PYENV_ROOT="${PYENV_ROOT:-${PYENV_ROOT_OSX}}" PATH="${PYENV_ROOT}/shims:${PATH}" AWS_CLI_ROOT="${HOME}/.aws_cli" AWS_ACCESS_KEY_ID__TO_BE_REEXPORTED_ON_DEPLOYS=AKIAV6A6G7RQ2HFZ5KP7 AWS_SECRET_ACCESS_KEY=[secure] RUST_BACKTRACE="all" BOOTSTRAPPED_PEX_KEY_SUFFIX=py36.linux PANTS_REMOTE_CA_CERTS_PATH=/usr/lib/google-cloud-sdk/lib/third_party/grpc/_cython/_credentials/roots.pem CACHE_NAME=python_tests.py36) 0
0.0
 Travis Job 36000.5
Source Files on build 36000
Detailed source file information is not available for this build.