uber / h3-go / 37044123883

100%

push

github

h3go: round degree conversions before the haversine subtraction (#154)

GreatCircleDistanceRads converts each coordinate from degrees to radians
and then subtracts. The Go spec lets the compiler fuse a product into a
following subtraction, and on arm64 it does, so one side of the
subtraction was exact and the other rounded. Identical points therefore
measured a few 1e-17 radians apart instead of exactly zero, and every
other distance picked up a last-bit difference from the C library, which
takes radians and has nothing to fuse.

Wrapping the four products in explicit float64 conversions rounds them
before use, which the spec guarantees prevents fusion. The zero-distance
test now requires exactly zero, including the point that exposed the
difference. The remaining fused instruction, in the sum of squares, matches
what a C compiler contracts within an expression and does not affect the
zero case.

Benchmarks are unchanged within noise: ten interleaved rounds of the old
and new binaries put every row, including the cgo and CellAreaRads2
controls, within 3% in both directions.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

9 of 9 new or added lines in 1 file covered. (100.0%)

4533 of 4533 relevant lines covered (100.0%)

2660413.48 hits per line