37452
0%
main: 80%

Ran 04 Feb 2021 01:03AM UTC

Jobs 1

Files 418

Run time 1min

Badge

Embed ▾

pending completion

Build # 37452

Build Type

push

travis-ci-com

Committed by

web-flow

Commit Message

Fix excessive mmap-ing in LMDB (cherrypick of #11513) (#11516)

### Problem

Core dumps collected on macOS are very large currently because they include all mmap'd data. According to `vmmap`, this is because LMDB is `mmap`'ing 1.7TB of data. And... it is! But not intentionally.

### Solution

`ShardedLmdb` was not dividing the `max_size` by the number of shards, and since each shard used their own LMDB `Environment`, we were ending up with `16 * max_size` `mmap`'d. Additionally, we had set some of our databases to sizes that are well beyond what we expect the database to grow to when GC is running consistently.

### Result

It is now possible to gather core dumps on macOS, which should assist in the debugging of #11364.

[ci skip-build-wheels]

Run Details

0 of 29050 relevant lines covered (0.0%)

0.0 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
5	37452.5 (PANTS_CONFIG_FILES="${TRAVIS_BUILD_DIR}/pants.travis-ci.toml" PANTS_DYNAMIC_UI=false LC_ALL="en_US.UTF-8" AWS_BUCKET=ci-public.pantsbuild.org BOOTSTRAPPED_PEX_KEY_PREFIX=daily/${TRAVIS_BUILD_NUMBER}/${TRAVIS_BUILD_ID}/pants.pex NATIVE_ENGINE_SO_KEY_PREFIX=monthly/native_engine_so MACOS_PYENV_PY27_VERSION=2.7.18 MACOS_PYENV_PY37_VERSION=3.7.7 MACOS_PYENV_PY38_VERSION=3.8.3 PYENV_ROOT_OSX=${HOME}/.pants_pyenv PYENV_ROOT="${PYENV_ROOT:-${PYENV_ROOT_OSX}}" PATH="${PYENV_ROOT}/shims:${PATH}" AWS_CLI_ROOT="${HOME}/.aws_cli" AWS_ACCESS_KEY_ID__TO_BE_REEXPORTED_ON_DEPLOYS=AKIAV6A6G7RQ2HFZ5KP7 AWS_SECRET_ACCESS_KEY=[secure] RUST_BACKTRACE="all" BOOTSTRAPPED_PEX_KEY_SUFFIX=py37.linux CACHE_NAME=python_tests.py37)	04 Feb 2021 01:03AM UTC	0	0.0	Travis Job 37452.5

Source Files on build 37452

Detailed source file information is not available for this build.

pantsbuild / pants / 37452
0%
main: 80%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 37452

pantsbuild / pants / 37452 0% main: 80%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 37452

pantsbuild / pants / 37452
0%
main: 80%

README BADGES
x