stillwater-sc / universal / 34775761975

86%

master: 84%

Build Type

push

github

Committed by web-flow

Commit Message

fix(blockdecimal): compare BID magnitudes as unsigned integers (#1481)

blockdecimal's BID encoding stores the magnitude as an Unsigned blockbinary and
ordered two magnitudes with blockbinary's operator<. That operator reads the top
storage bit as a sign bit whatever the BinaryNumberType is (#1479), so every BID
magnitude >= 2^(nbits-1) compared as negative. The ordering of blockdecimal<3,BID>
was wrong for 998,448 of its 3,996,001 signed pairs (25.0%).

Long division uses that comparison to decide whether the remainder still holds the
divisor, so it exited early or never exited: 518/52 gave 0, 600/700 gave 116, and
1/512 did not terminate. % inherited the same failures.

dfixpnt picks the sign of a mixed-sign sum or difference, and its saturation
clamp, from the same comparison. Of 200,000 random dfixpnt<8,4,BID> pairs,
44,032 sums and 44,177 differences were wrong, and so were 44,177 orderings.

less_than_magnitude() now compares BID magnitudes via bb_to_uint64(); BID is
limited to 19 digits, so both fit. BCD and DPD keep the digit-by-digit compare.
blockbinary is unchanged: posit, lns, takum, quire and dfloat also use unsigned
blockbinaries, and #1479 audits them.

maxval() now clears the storage first. For BCD and DPD it set digits one at a
time into whatever storage it was given, so the bits above the top digit kept
their previous contents. dfixpnt's saturation clamp, maxpos() and maxneg() call it
on uninitialized blockdecimals, and operator== compares whole blocks. gcc flagged
the uninitialized read (-Wmaybe-uninitialized) once the new test instantiated
dfixpnt<9,4,BCD,Saturate>. The dfixpnt and blockdecimal differentials against main
are byte-identical with and without this line.

Three regression tests are added:
- internal/blockdecimal/arithmetic/division.cpp (bd_division): quotient and
  remainder against C++ / and % for every encoding, both signs, with the top
  storage bit set at 3, 6, 9 and 19 digits. Exhaustive over 3 digits at level 3.
... (continued)

Coverage Stats

102 of 134 new or added lines in 3 files covered. (76.12%)

13 existing lines in 2 files now uncovered.

43476 of 50735 relevant lines covered (85.69%)

7327069.04 hits per line