dignifiedquire / sonora / 37903767722

96%

push

github

perf: use C++-order arithmetic except on AArch64 and x86 with fma (#37)

* perf: use C++-order arithmetic except on AArch64 and x86 with fma

ac75be6 rewrote 41 multiply-add expressions as f32::mul_add chains:
the butterflies and real-FFT post-processing in fft4g (cft1st, cftmdl,
rftfsub, rftbsub), the cascaded biquad filter, and the unrolled taps of
the three-band filter bank. mul_add is one instruction only where the
target has FMA. On x86_64 without the fma target feature, which is the
default for every downstream build, each mul_add becomes an fmaf
library call: on Linux an indirect call through the GOT to
compiler_builtins' runtime-dispatched fmaf, on Windows MSVC the UCRT
import. These calls sit in per-sample loops: the biquad feeds the
high-pass filter and the AEC3 decimator, and the filter bank and fft4g
run on every frame.

With a static CRT on Windows (+crt-static), the result of libucrt's
fmaf is not flushed under FTZ/DAZ. On the #34 flush-to-zero branch
(fix/issue-34-denormals), its regression test finds 503 subnormal
values in the AEC3 decimator output; with this change it passes.

Each site now selects its form with a compile-time constant,
USE_FMA = cfg!(any(target_arch = "aarch64", target_arch = "arm64ec",
target_feature = "fma")). Those targets keep the mul_add chains
unchanged; arm64ec runs AArch64 code and compiles mul_add to fmadd, as
aarch64 does. Every other target uses the expressions from before
ac75be6, which follow the operation order of the C++ reference (M145);
GCC and clang do not fuse them on baseline x86_64. Other targets with
native FMA (riscv64gc, powerpc64, armv7 with VFPv4, and others) also
take the plain path. It follows the C++ source order, as C++ compiled
without FP contraction does; GCC and clang would fuse these
expressions there.

Verification:
- aarch64 output is bit-identical to main. Release assembly of the
  three crates is instruction-identical after normalizing symbol
  hashes, and a pipeline run (HPF, A... (continued)

604 of 604 new or added lines in 3 files covered. (100.0%)

30444 of 31821 relevant lines covered (95.67%)

6457947.19 hits per line