petl-developers / petl / 33407171861

92%

push

github

Fix lookupjoin TypeError on empty tables and uncomparable keys (#708)

* Fix lookupjoin TypeError on empty tables and uncomparable keys

iterlookupjoin compared join keys with operator.itemgetter and plain
None sentinels, so the merge loop raised TypeError whenever a key
comparison crossed types. iterjoin and iterantijoin were switched to
comparable_itemgetter and Comparable(None) in 1363d93; apply the same
change to the third merge loop.

* fix: keep the first left group in lookupjoin when its key is None

The post-loop guard in iterlookupjoin used `lkval > rkval` to decide
whether a left row group had been picked off lgit but not yet yielded.
That encodes "iterator exhausted" as a key value, and None is a valid
key, so a left table whose first key is None joined against an empty
right table left both sides at Comparable(None). The guard compared
equal, the pending group was skipped, and advancing lgit invalidated
it - dropping a left row from what is documented as a left join.

Track the pending group with an explicit boolean instead, so no key
value has to double as a sentinel.

Before:
    lookupjoin((('id','x'), (None,1), (2,2)), (('id','y'),), key='id')
    -> ('id','x','y'), (2,2,None)
After (and matching hashlookupjoin):
    -> ('id','x','y'), (None,1,None), (2,2,None)

Covered by two new cases in _test_lookupjoin_empty, which run against
both lookupjoin and hashlookupjoin.

---------

Co-authored-by: Dylan Pulver <dylanpulver@users.noreply.github.com>

44 of 44 new or added lines in 2 files covered. (100.0%)

14183 of 15411 relevant lines covered (92.03%)

0.92 hits per line