linkedin / Burrow / 73

74%

push

travis-ci

Store broker offset history (#338)

* Add a ring of broker offsets for each partition to keep a short history of partition LEO

* Clean up repetitive casting in tests

* Add asserts to check for broker offset history

* Delay alerting for stopped partition

In order to account for the race condition described in #303 (second part), we need to delay alerting for a stopped partition for a short period of time. We do that by only marking the partition stopped if the timestamps show it is stopped AND if the partition did not have zero lag against any recent broker LEO. In the case where the intervals config for storage is 10 and the cluster offset-refresh is 30 seconds, this would give the consumer 5 minutes to consume from a slow partition and commit an offset before it gets marked as stopped.

* Update current tests for the evaluator to handle the new recent lag check

* Reformat test objects so they're readable

* Add a test to cover a slow data partition as described in #303

* gofmt fixes

37 of 37 new or added lines in 2 files covered. (100.0%)

2998 of 4061 relevant lines covered (73.82%)

68.29 hits per line