brawer / osmdiffs / 43048112767

94%

push

github

feat(pipeline): write and publish spider-stats.parquet (#936)

A new step, compute_spider_stats, scans conflated.parquet once and
writes one row per AllThePlaces spider: what became of its features,
how they matched (by relation, rule and a distance histogram), how far
their tags agree with OpenStreetMap, and per country and Wikidata ID,
how its brands compare with OpenStreetMap's totals and deletion
candidates. It adds the side files of import_atp and
collect_wikidata_ids and AllThePlaces' own run statistics.

The file is uploaded and listed in datapackage.json as a debugging
aid, and documented in docs/outputs/SPIDER_STATS.md. Part of #924.


Claude-Session: https://claude.ai/code/session_01Ttfb2LhDRExioXkdHbxLno

Co-authored-by: Claude <noreply@anthropic.com>

701 of 716 new or added lines in 7 files covered. (97.91%)

18546 of 19710 relevant lines covered (94.09%)

3396.44 hits per line