ghga-de / hexkit / 10563191601

93%

main: 93%

push

github

Kafka DLQ  (#123)

* Expand KafkaConfig for DLQ

* Add DLQ functionality

* Remove reference to Self

* Fix weird formatting

* Add caplog record checking util functions

* Improve pre-consume validation

Rename ExtractedEvent to ExtractedEventInfo

* Delete unused get_header_value & its error

Move OriginalTopicError into DLQ subscriber only

* Improve logging and documentation

* Update tests to check logs, add more cases, and consolidate

* Rename 'publisher' attr to 'dlq_publisher'

* Add 'title=' to new config

* Change lowercase dlq to DLQ

* Make assorted documentation typo fixes

* Missed documentation

* Add configurable exponential backoff for direct retries

* Validate retries_left param in _retry_event()

Improve error handling in _handle_consumption()

* Add headers to event pub protocol/move og topic to headers

* Adapt tests for updated protocol and dlq header changes

* Remove unused ignores

* Make mypy happy again

* Add headers param to kafka fixture publish_event

* Allow callbacks in KafkaDLQSubscriber

* Make ExtractedEventInfo more useful

* Remove redundant header check

Co-authored-by: Christoph Zwerschke <c.zwerschke@dkfz-heidelberg.de>

* Fix some doc strings

* Move consumer.start() outside of try block

* Tweak error messages

* Simplify list comprehension in validate_dlq_headers

* Add Dockerfile.debian to ignored list

* Use RESERVED_HEADERS const and remove lead underscore from og topic

* Add RetriesLeftError

* Update src/hexkit/providers/akafka/provider/eventsub.py

Co-authored-by: Christoph Zwerschke <c.zwerschke@dkfz-heidelberg.de>

* Clarify an error message in _validate_extracted_info()

* Fix some wording issues

* Fix a test and remove debug print statements

---------

Co-authored-by: Christoph Zwerschke <c.zwerschke@dkfz-heidelberg.de>

218 of 224 new or added lines in 9 files covered. (97.32%)

1944 of 2107 relevant lines covered (92.26%)

0.92 hits per line