dangernoodle-io / breadboard / 29887567633

96%

push

github

fix: derive host-test verdict from a direct binary run, not pio's async test reader (B1-1137) (#990)

PlatformIO 6.1.19's NativeTestOutputReader (platformio/test/runners/readers/native.py) has two upstream defects, confirmed by direct reproduction against this repo's own native env, not by reading pio's source alone:

Bug A: raise_for_status() calls signal.Signals(abs(return_code)) on any nonzero return code with no check for signal death (negative returncode) vs a normal exit(N) (positive). Unity's main() returns UNITY_END(), the FAILURE COUNT, as its exit status, so one failing test exits 1 and gets printed as 'Program received signal SIGHUP (Hangup: 1)' instead of '1 test failed'. Reproduced live: injecting one deliberate TEST_FAIL into test_bb_str.c (reverted, not committed) and running the unmodified 'pio test -e native' path prints exactly that misleading SIGHUP line.

Bug B: under 'pio test', the child's stdout is a pipe, so libc fully buffers it and the tail (including Unity's own summary line) flushes only at process exit; pio's asyncio reader ends the run on process_exited(), racing the final pipe_data_received() delivery of that buffered tail, and can lose the summary line entirely.

CI runs the identical path: 'make coverage' depends on 'make test', and both invoke 'pio test -e native{,_lock_stats_off}' via scripts/coverage_toolchain.sh, which only PATH-shims gcc/g++/gcov and forwards the child's exit status -- so CI shares this exposure, not just local runs.

New verdict path (scripts/run_host_tests.py, wired into 'make test' for both native envs): 'pio test' still builds and prints its human-readable per-case report, but its TEST verdict is ignored; pass/fail instead comes from a plain synchronous subprocess.run() of the already-compiled Unity binary at .pio/build/<env>/program, cross-checked against Unity's own '<N> Tests <M> Failures <P> Ignored' summary line -- a missing summary line, a zero-tests-executed summary, or a run exceeding ... (continued)

7458 of 8054 branches covered (92.6%)

Branch coverage included in aggregate %.

12619 of 12781 relevant lines covered (98.73%)

2421.45 hits per line