OCHA-DAP / hdx-scraper-ophi / 33718718487

94%

main: 97%

push

github

Simplify inlined failure logs in summarize_country_results

Two fixes, both scoped to the summary step's failure inlining:

- retriever_context() left delete_on_failure at temp_dir_batch()'s
  default of True, so any task failure deleted the shared scratch folder
  out from under sibling tasks still using it - a real 429-rate-limited
  failure was being masked by a spurious FileNotFoundError from a
  different task's cleanup racing on the same folder. Fixed alongside
  the existing delete_scratch_on_success=False default, same reasoning.

- summarize_country_results was inlining a failed instance's log via a
  bare log_file.read_text(), dumping the raw JSON-lines file verbatim -
  every line's context keys (logger, ti_id, try_number, map_index,
  dag_id, task_id, run_id, filename, lineno) plus, for the actual
  failure record, a full frame-by-frame traceback for every exception in
  the chain. Added _render_task_log() to reformat this: plain
  "<timestamp> <LEVEL> <event>" lines with the redundant per-line keys
  dropped, and a "Task failed with exception" record's error_detail
  rendered as a short exception-chain summary (exc_type: exc_value per
  exception) instead of the full stack frames - the actual error is now
  immediately visible instead of buried under stack-trace noise.

15 of 19 new or added lines in 1 file covered. (78.95%)

545 of 581 relevant lines covered (93.8%)

0.94 hits per line