pantsbuild / pants / 9185
0%
main: 93%

LAST BUILD BRANCH: 2.33.x
DEFAULT BRANCH: main
Ran
Jobs 7
Files 0
Run time –
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
9185

push

travis-ci

stuhood 
[engine] Introduce content-addressability

This review converts all Subject values to SubjectKeys, and stores subjects in a dumb content-addressable dictionary. The idea is that in the longer term, subject values go in local databases like sqlite/leveldb/etc (see [#2968](https://github.com/pantsbuild/pants/issues/2968)), in caches, etc.

- Introduce SubjectKey which is `eq`'d/`hash`'d using only the digest it contains. It additionally optionally contains the `str` representation of the Subject, for debugging purposes.
- Remove the lazily-memoized hash from Address in order to make it internally immutable, and eagerly hash it. Since Adresses are almost always used in sets/collections, this should not have a performance impact.
- Add a temporary solution to ignore dotfiles in the `fs` package (until we dive into [#2956](https://github.com/pantsbuild/pants/issues/2956)).
- Generate BuildRequests with `scheduler.build_request(goals, subjects)`, which handles storing the initial subjects and generating roots.
- Remove some of the magic around `type_alias`, and store it in a Struct whenever it exists.
- Rename the object that is passed to `Node.step` to `StepContext`, and make the NodeBuilder concept private to the Scheduler.
- Introduce a StatefulPool multiprocessing pool, which allows processes to hold/mutate their own copies of a value between items processed by the pool.
    - This is used here by LocalMultiprocessEngine to have workers with their own copy of the NodeBuilder, and their own Subjects collections. In some regards, this serves to make the LocalMultiprocessEngine even _more_ of a toy, because a real implementation will likely use a local database or shared memory to store.
- Pass Subjects introduced by Steps back to the Scheduler so that they can be stored in the Subjects instance.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/112062727

Bugs closed: 2966, 2971

Reviewed at https://rbcommons.com/s/twitter/r/3498/
Jobs
ID Job ID Ran Files Coverage
1 9185.1 (CI_FLAGS="-cjlpn 'Various pants self checks'") 0
Travis Job 9185.1
4 9185.4 (CI_FLAGS="-fkmsrcjlp 'Python contrib tests'") 0
Travis Job 9185.4
5 9185.5 (CI_FLAGS="-fkmsrjlpn -i 0/6 'Python integration tests for pants - shard 1'") 0
Travis Job 9185.5
6 9185.6 (CI_FLAGS="-fkmsrjlpn -i 1/6 'Python integration tests for pants - shard 2'") 0
Travis Job 9185.6
7 9185.7 (CI_FLAGS="-fkmsrjlpn -i 2/6 'Python integration tests for pants - shard 3'") 0
Travis Job 9185.7
9 9185.9 (CI_FLAGS="-fkmsrjlpn -i 4/6 'Python integration tests for pants - shard 5'") 0
Travis Job 9185.9
10 9185.10 (CI_FLAGS="-fkmsrjlpn -i 5/6 'Python integration tests for pants - shard 6'") 0
Travis Job 9185.10
Source Files on build 9185
Detailed source file information is not available for this build.