ExpediaGroup / drone-fly / #146

36%

master: 59%

Build Type

push

Committed by web-flow

Commit Message

feat: allow the Kafka record key deserializer to be configured (#59)

Drone Fly reads record keys with a LongDeserializer, the default in the Apiary
KafkaMessageReader, which matches the key written by the Apiary Hive Metastore
listener. A topic populated by a different producer may key its records with
another type, and reading those fails on every record with:

  SerializationException: Size of data received by LongDeserializer is not 8

The consumer offset does not advance past a record it cannot deserialize, so
Drone Fly retries the same offset indefinitely and stops processing entirely.

Resolve the key deserializer from the consumer properties and pass it to the
builder explicitly, defaulting to LongDeserializer so existing deployments are
unaffected. It cannot be picked up from withConsumerProperties, because those
values do not override the builder's own defaults; apiary-extensions 8.2.5 adds
withKeyDeserializer for this purpose.

The record key is not used to process the event, so any deserializer that can
read the key is safe.

Co-authored-by: javbeltran_expedia <javbeltran@expediagroup.com>
Co-authored-by: Claude Opus <noreply@anthropic.com>

Coverage Stats

2 of 3 new or added lines in 1 file covered. (66.67%)

7 existing lines in 1 file now uncovered.

136 of 381 relevant lines covered (35.7%)

0.36 hits per line