8843
0%
main: 93%

Ran 29 Jan 2016 11:33PM UTC

Jobs 7

Files 0

Run time –

Badge

Embed ▾

pending completion

Build # 8843

Build Type

push

travis-ci

Committed by

peiyuwang

Commit Message

Allow failover for remote cache

Today pants can have multiple remote cache hosts either through config or
through the *resolve* process, within each task pants will first decide a
*best* url and stick with the same url throughout the task run.

The problem is if this url dies there is no failover even though other urls are
still be available. The cache instances dying turns out to be not that
uncommon, most recent incident happened in Twitter was due to a memory leak
that was made worse by bloated artifacts so hosts died a few times a day. When
this happens we see pants default to local compile that takes forever.

This review does not attempt to implement load balancing, it does not even try
to re-ping (because running pinger is expensive) it only tries to address the
very specific problem: individual host dying during the single pants run and
allow pants to fail over to 2nd or 3rd instance.

It does not address if all *urls* die but as long as there is one instance alive
now pants can eventually fail over.

Testing Done:
https://travis-ci.org/peiyuwang/pants/builds/104828349

Bugs closed: 2837

Reviewed at https://rbcommons.com/s/twitter/r/3374/

Coverage Stats

Jobs

ID	Job ID	Ran
1	8843.1 (CI_FLAGS="-cjlpn 'Various pants self checks'")	29 Jan 2016 11:33PM UTC	Travis Job 8843.1
4	8843.4 (CI_FLAGS="-fkmsrcjlp 'Python contrib tests'")	29 Jan 2016 11:51PM UTC	Travis Job 8843.4
5	8843.5 (CI_FLAGS="-fkmsrjlpn -i 0/6 'Python integration tests for pants - shard 1'")	29 Jan 2016 11:40PM UTC	Travis Job 8843.5
6	8843.6 (CI_FLAGS="-fkmsrjlpn -i 1/6 'Python integration tests for pants - shard 2'")	29 Jan 2016 11:44PM UTC	Travis Job 8843.6
7	8843.7 (CI_FLAGS="-fkmsrjlpn -i 2/6 'Python integration tests for pants - shard 3'")	29 Jan 2016 11:46PM UTC	Travis Job 8843.7
8	8843.8 (CI_FLAGS="-fkmsrjlpn -i 3/6 'Python integration tests for pants - shard 4'")	29 Jan 2016 11:45PM UTC	Travis Job 8843.8
10	8843.10 (CI_FLAGS="-fkmsrjlpn -i 5/6 'Python integration tests for pants - shard 6'")	29 Jan 2016 11:55PM UTC	Travis Job 8843.10

Source Files on build 8843

Detailed source file information is not available for this build.

pantsbuild / pants / 8843
0%
main: 93%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 8843

pantsbuild / pants / 8843 0% main: 93%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 8843

pantsbuild / pants / 8843
0%
main: 93%

README BADGES
x