uber / h3-go / 37685604085

100%

push

github

fix: round degree conversions before the haversine subtraction (#161)

GreatCircleDistanceRads in the root package is pure Go. It converts each
coordinate from degrees to radians and then subtracts, and the Go spec
lets the compiler fuse a product into a following subtraction. On arm64
it does, so one side of the subtraction was exact and the other rounded:
identical points measured a few 1e-17 radians apart instead of exactly
zero, and every other distance picked up a last-bit difference from the
C library, which takes radians and has nothing to fuse. x/h3go had the
same defect and the same fix in an earlier change; the root package was
missed because nothing ran its tests on arm64 until the differential
paritytest package was run on macOS, where
TestGreatCircleDistanceMatchesCgo fails for every point against itself.

Wrap the four products in explicit float64 conversions, which the spec
guarantees rounds them before use and so prevents the fusion. Add a
test that requires exactly zero for a point against itself, including
the points that exposed the difference. amd64 is unaffected.

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

9 of 9 new or added lines in 1 file covered. (100.0%)

4545 of 4545 relevant lines covered (100.0%)

4168090.13 hits per line