pantsbuild / pants / 36839
0%
main: 80%

LAST BUILD BRANCH: gha-deps
DEFAULT BRANCH: main
Ran
Jobs 1
Files 429
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
36839

push

travis-ci-com

web-flow 
Clean the graph speculatively, and cancel nodes when interest is lost (cherrypick of #11308) (#11310)

### Problem

#11290 describes a case where attempting to clean a (relatively) long-running uncacheable node (a test that has failed) causes us to run the test twice. This is caused by graph cleaning: the attempt to clean the graph requests all of the node's previous dependencies in parallel, but waits until they have all returned before determining whether the node could be cleaned or not. Concretely, this meant that although the sources of the test had changed, we would re-run the (uncacheable) failed test again to completion before noticing the changed sources, and running it a second time with the changed sources.

### Solution

As mentioned on #11290, [other monadic build systems](https://neilmitchell.blogspot.com/2020/11/data-types-for-build-system-dependencies.html) have taken the approach of sequencing requests for dependencies during cleaning, rather than requesting them in parallel. In this case, that would mean sequentially requesting the sources of the node before running the test. But because Pants uses future based concurrency aggressively, it is more challenging for us to preserve the structure of an `@rule` `MultiGet` all the way down into a single request to the `Graph`: without custom tuple-based combinators, we would lose type safety when we needed to request a `Vec<Node<_>>` rather than `future::join_all`ing a series of type-safe requests.

Instead, this change takes a different strategy, and [speculates](https://en.wikipedia.org/wiki/Speculative_execution) while cleaning. All dependencies are still requested in parallel, but we now fail fast as soon as we detect a mismatch, and all speculative branches that were incorrect are canceled. When compared to sequentially requesting dependencies, this should almost always be faster (*as long as the total number of processes being cleaned is smaller than the constraints we have on... (continued)

0 of 29928 relevant lines covered (0.0%)

0.0 hits per line

Jobs
ID Job ID Ran Files Coverage
5 36839.5 (PANTS_CONFIG_FILES="${TRAVIS_BUILD_DIR}/pants.travis-ci.toml" PANTS_DYNAMIC_UI=false LC_ALL="en_US.UTF-8" AWS_BUCKET=ci-public.pantsbuild.org BOOTSTRAPPED_PEX_KEY_PREFIX=daily/${TRAVIS_BUILD_NUMBER}/${TRAVIS_BUILD_ID}/pants.pex NATIVE_ENGINE_SO_KEY_PREFIX=monthly/native_engine_so PYENV_PY27_VERSION=2.7.18 PYENV_PY36_VERSION=3.6.10 PYENV_PY37_VERSION=3.7.7 PYENV_PY38_VERSION=3.8.3 PYENV_ROOT_OSX=${HOME}/.pants_pyenv PYENV_ROOT="${PYENV_ROOT:-${PYENV_ROOT_OSX}}" PATH="${PYENV_ROOT}/shims:${PATH}" AWS_CLI_ROOT="${HOME}/.aws_cli" AWS_ACCESS_KEY_ID__TO_BE_REEXPORTED_ON_DEPLOYS=AKIAV6A6G7RQ2HFZ5KP7 AWS_SECRET_ACCESS_KEY=[secure] RUST_BACKTRACE="all" BOOTSTRAPPED_PEX_KEY_SUFFIX=py36.linux PANTS_REMOTE_CA_CERTS_PATH=/usr/lib/google-cloud-sdk/lib/third_party/grpc/_cython/_credentials/roots.pem CACHE_NAME=python_tests.py36) 0
0.0
 Travis Job 36839.5
Source Files on build 36839
Detailed source file information is not available for this build.