GoogleCloudPlatform / google-cloud-java / 3902

72%

Pull #2055

travis-ci

pubsub: start connections slowly

Starting a new connection seems to add a task which blocks
a thread in our thread pool.
It does not block for long, but Subscriber starts many connections,
slamming the pool.
If the number of threads the pool is low, the pool is blocked for
a long time, causing strange delays and RPC failures.

This PR alleviates this problem by starting connections one at a time.
- Each connection now make one successful call before declaring itself running.
- The Subscriber waits for a connection to be in running state
  before starting another.

Together, these changes limit the number of blocking tasks
registered to the pool.

Since starting the Subscriber can now take a significant amount of time,
and `doStart` should return promptly,
a new thread is used to start up the connections.

19 of 19 new or added lines in 2 files covered. (100.0%)

19197 of 23730 relevant lines covered (80.9%)

139.84 hits per line