tarantool / test-run / 9285128968

63%

master: 63%

Build Type

Pull #434

github

Committed by Totktonada

Commit Message

dispatcher: lift pipe buffer size restriction

A task queue dispatcher puts all the tasks to the task queue at startup.
Then workers are started and are taking the tasks from it.

If there are many tasks in a task group (which roughly corresponds to a
test suite), we can reach the pipe buffer size on putting into the
queue, because `multiprocessing.SimpleQueue` uses a pipe under the hood.

The solution is to use `multiprocessing.Queue`, which has an
intermediate buffer before the underlying pipe and writes to the pipe in
a background thread, without blocking a thread that calls
`<queue>.put()`.

The `Queue` API is a superset of the `SimpleQueue` API, so we can just
replace the implementation.

Let's also use `Queue` for the worker's output queue to be on the safe
side and for consistency.

Fixes #287

Pull Request Pull Request #434: dispatcher: lift pipe buffer size restriction

Run Details

760 of 1564 branches covered (48.59%)

Branch coverage included in aggregate %.

3 of 4 new or added lines in 1 file covered. (75.0%)

1 existing line in 1 file now uncovered.

2938 of 4349 relevant lines covered (67.56%)

0.68 hits per line