Netflix / conductor / 4791

70%

master: 70%

push

travis-ci-com

Improve postgresql DAO performance (#1940)

* Improve postgresql DAO performance

by adding proper FOR SHARE / FOR UPDATE / SKIP LOCKED locks to DB queries
where it makes sense in order to reduce conflicts/deadlocks in DB.

Most important case is when workers poll for work, they are not
interested in tasks that are locked (currently being updated) so they
can leverage 'SKIP LOCKED' to prevent DB locks and tx deadlocks.

This increases the performance of postgres dao

Additional improvements:

+ move thread.sleep out of DB transaction when tasks are being polled
with timeout parameter

+ add serialization_error to the list of causes triggerring TX retry
(this is happening under heavy load)

+ fix processUnack condition: it used to do the opposite. This has been
also fixed in the meantime by u447 <rick.fishman@bcbsfl.com>

+ add a performance test. This test can be executed manually, but should
not be automated

+ add retries to containsMessage method

Signed-off-by: Maros Marsalek <mmarsalek@frinx.io>

* Remove double query from processUnacks

That was a mistake in the original commit, no need to do the same thing twice

77 of 77 new or added lines in 3 files covered. (100.0%)

13025 of 18562 relevant lines covered (70.17%)

0.7 hits per line