broadinstitute / catch / 282

94%

push

travis-ci-com

Close probe finding pool even if no work is submitted

Previously, probe.close_probe_finding_pool() only closed
the multiprocessing.Pool if work was submitted to the pool
(see previous comment for why -- this was to avoid a bug
in Python 2.7).

This seems to be causing issues. Travis CI is sporadically
failing on the test `test_probe.test_too_short_sequence_large_k`
with `OSError: [Errno 12] Cannot allocate memory`. This test
uses very little memory. Instead, this is probably occurring
because creating a multiprocessing.Pool (when opening the
probe finding pool) is unable to call `os.fork()` due to there
being too many open file descriptors. Many file descriptors are
not closed if no work was submitted to an open pool, because
pool.close_probe_finding_pool() deliberately did not close
its pool in this case. This unit test, as well as others,
open probe finding pools but do not submit any work to them.
As a result, they leave open file descriptors.

We no longer have to worry about the bug in Python 2.7 because
we test only in Python 3. This change attempts to fix the
sporadic failures in Travis CI.

1416 of 1599 branches covered (88.56%)

1 of 1 new or added line in 1 file covered. (100.0%)

4858 of 5107 relevant lines covered (95.12%)

2.85 hits per line