5835
70%

Ran 09 Nov 2018 05:56PM UTC

Jobs 4

Files 383

Run time 36min

Badge

Embed ▾

pending completion

Build # 5835

Build Type

push

travis-ci

Committed by

hroberts

Commit Message

once fetch link queue gets below 25 links, requeue and exit

the fetch_link queue works in chunks of 10k urls.  this change makes
the parent process just requeue everything without domain throttling
once the queue gets below 25.  this is to speed up the last few urls
in the queue, which can take a long time to complete due to getting
starved out by the domain throttle scheduling and also prevents a couple
of failed fetch link jobs from killing the whole topic spidering job.

Run Details

14913 of 22736 relevant lines covered (65.59%)

286.05 hits per line

Jobs

ID	Job ID	Ran	Coverage
1	5835.1 (MC_TEST_CHUNK=1)	09 Nov 2018 05:56PM UTC	58.07	Travis Job 5835.1
2	5835.2 (MC_TEST_CHUNK=2)	09 Nov 2018 06:09PM UTC	52.38	Travis Job 5835.2
3	5835.3 (MC_TEST_CHUNK=3)	09 Nov 2018 06:12PM UTC	53.73	Travis Job 5835.3
4	5835.4 (MC_TEST_CHUNK=4)	09 Nov 2018 06:14PM UTC	54.91	Travis Job 5835.4

Source Files on build 5835

Detailed source file information is not available for this build.

berkmancenter / mediacloud / 5835
70%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 5835

berkmancenter / mediacloud / 5835 70%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 5835

berkmancenter / mediacloud / 5835
70%

README BADGES
x