pantsbuild / pants / 8142
0%
main: 93%

LAST BUILD BRANCH: csb/locks-toggle-0
DEFAULT BRANCH: main
Ran
Jobs 8
Files 0
Run time –
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
8142

push

travis-ci

stuhood 
Make monolithic jars produced by bundle/binary slimmer

Monolithic jar does not scale, even with --no-deployjar that excludes external
jars, internal dependencies can still be large. In Twitter we saw a few hundred
Mbytes jars to transfer, for remote offices this is more painful. Breaking into
smaller jars can also benefit hadoop folks to better utilize their caching layer [1]

This review makes --no-deployjar no longer bundle internal libraries, instead,
make them symlinks under libs like we do for 3rdparty, the links will go to those
jars under .pants.d, symlinks will have stable names, those stable names are used
as archive contents, they are also to be embeded as the Manifest's Class-Path
attribute.

Current --no-deployjar
```
app-bundle.jar // this can be large
libs/
libs/3rdparty-x.jar
libs/3rdparty-others.jar
```
New --no-deployjar
```
app-bundle.jar // this only sets Class-Path + other Manifest entries
libs/
libs/3rdparty-x.jar
libs/3rdparty-other.jar
libs/internal-library-a.jar // symlinks to .pants.d, unless it's shaded
libs/internal-library-other.jar
```
---deployjar will remain as is.

[1] Hadoop use case is they use SharedCache distribute jars from HDFS. They have
    a caching layer that does file checksums so if we break down into smaller jars
    they will be able to transfer less.
    http://www.slideshare.net/ctrezzo/hadoop-summit2015-yarnsharedcachev12

Testing Done:
https://travis-ci.org/peiyuwang/pants/builds/91439042

Bugs closed: 2577

Reviewed at https://rbcommons.com/s/twitter/r/3133/
Jobs
ID Job ID Ran Files Coverage
1 8142.1 (CI_FLAGS="-cjlpn 'Various pants self checks'") 0
Travis Job 8142.1
4 8142.4 (CI_FLAGS="-fkmsrcjlp 'Python contrib tests'") 0
Travis Job 8142.4
5 8142.5 (CI_FLAGS="-fkmsrjlpn -i 0/6 'Python integration tests for pants - shard 1'") 0
Travis Job 8142.5
6 8142.6 (CI_FLAGS="-fkmsrjlpn -i 1/6 'Python integration tests for pants - shard 2'") 0
Travis Job 8142.6
7 8142.7 (CI_FLAGS="-fkmsrjlpn -i 2/6 'Python integration tests for pants - shard 3'") 0
Travis Job 8142.7
8 8142.8 (CI_FLAGS="-fkmsrjlpn -i 3/6 'Python integration tests for pants - shard 4'") 0
Travis Job 8142.8
9 8142.9 (CI_FLAGS="-fkmsrjlpn -i 4/6 'Python integration tests for pants - shard 5'") 0
Travis Job 8142.9
10 8142.10 (CI_FLAGS="-fkmsrjlpn -i 5/6 'Python integration tests for pants - shard 6'") 0
Travis Job 8142.10
Source Files on build 8142
Detailed source file information is not available for this build.