evolution-gaming / kafka-journal / 14332208462

83%

master: 94%

Build Type

Pull #739

Committed by migesok

Commit Message

New improved Kafka offset commit logic for replicator

The next release version should have a minor increment - 4.2.0

- TopicCommit.apply renamed to TopicCommit.sync, TopicCommit.apply deprecated
- Added TopicCommit.asyncPeriodic implementation
- TopicCommit.delayed deprecated in favor of TopicCommit.asyncPeriodic
- TopicCommit.asyncPeriodic is used by default in the replicator
- updated skafka for an async commit method fix

Main features of asyncPeriodic:
- periodic commits are done asynchronously without blocking the main poll loop, improving latency
- it properly handles rebalance, not loosing progress for revoked partitions
- same logic commits offsets on app shutdown because Java consumer invokes revoke partition callbacks
  on closure
- gracefully handles RebalanceInProgressException

Additionally:
- made TopicCommit journal package private - it is used only in journal package private APIs,
  and it is not supposed to be used outside the internal journal code
- ToTry[StateT] impl in ConsumeTopicTest - it didn't work on flatMap'ed RebalanceCallback's

Pull Request Pull Request #739: New improved Kafka offset commit logic for replicator

Run Details

61 of 78 new or added lines in 5 files covered. (78.21%)

18 existing lines in 14 files now uncovered.

3225 of 3885 relevant lines covered (83.01%)

1.4 hits per line