34890
0%
main: 93%

Ran 15 Aug 2020 03:13PM UTC

Jobs 1

Files 440

Run time 26s

Badge

Embed ▾

pending completion

Build # 34890

Build Type

push

travis-ci-com

Committed by

web-flow

Commit Message

Spawning against materialized binaries works. (#10577)

It's generically unsafe to fork+exec against binaries written out in a
multithreaded program when concurrent forks are possible. Even if all
files are opened for writing with O_CLOEXEC (which is the case for
Rust) if thread1 opens a file for writing and then thread2 forks,
thread2 will hold an open file descriptor. If thread2's subsequent exec
is delayed past the fork+exec thread1 does against the file it wrote,
then the thread1 fork+exec'd process will encounter ETXTBSY.

OSX "solves" this by retrying some number of times when it hits ETXTBSY,
but Linux does not attempt this hack. O_CLOFORK has been proposed and
adopted by some unices, but not Linux.

As such we need to make some tradeoff to allow this use case. This 
change introduces a lock around process spawns (fork+exec) to prevent
interleaved fork / exec whenever there is a spawn that we know exec's a 
binary we wrote out. Since forks can still happen in our process space
outside of our control (in libraries), this change also introduces a 
bounded ETXTBSY retry loop as a fallback for binaries we wrote out.

Fixes #10507

Coverage Stats

0 of 29061 relevant lines covered (0.0%)

0.0 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
5	34890.5 (PANTS_CONFIG_FILES="${TRAVIS_BUILD_DIR}/pants.travis-ci.toml" PANTS_DYNAMIC_UI=false LC_ALL="en_US.UTF-8" AWS_BUCKET=ci-public.pantsbuild.org BOOTSTRAPPED_PEX_KEY_PREFIX=daily/${TRAVIS_BUILD_NUMBER}/${TRAVIS_BUILD_ID}/pants.pex NATIVE_ENGINE_SO_KEY_PREFIX=monthly/native_engine_so PYENV_PY27_VERSION=2.7.18 PYENV_PY36_VERSION=3.6.10 PYENV_PY37_VERSION=3.7.7 PYENV_PY38_VERSION=3.8.3 PYENV_ROOT_OSX=${HOME}/.pants_pyenv PYENV_ROOT="${PYENV_ROOT:-${PYENV_ROOT_OSX}}" PATH="${PYENV_ROOT}/shims:${PATH}" AWS_CLI_ROOT="${HOME}/.aws_cli" AWS_ACCESS_KEY_ID__TO_BE_REEXPORTED_ON_DEPLOYS=AKIAV6A6G7RQ2HFZ5KP7 AWS_SECRET_ACCESS_KEY=[secure] RUST_BACKTRACE="all" BOOTSTRAPPED_PEX_KEY_SUFFIX=py36.linux PANTS_REMOTE_CA_CERTS_PATH=/usr/lib/google-cloud-sdk/lib/third_party/grpc/_cython/_credentials/roots.pem CACHE_NAME=python_tests.py36)	15 Aug 2020 03:13PM UTC	0	0.0	Travis Job 34890.5

Source Files on build 34890

Detailed source file information is not available for this build.

pantsbuild / pants / 34890
0%
main: 93%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 34890

pantsbuild / pants / 34890 0% main: 93%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 34890

pantsbuild / pants / 34890
0%
main: 93%

README BADGES
x