dbekaert / RAiDER / 465d368c-3137-46fb-9666-d5aa28a785ef

51%

dev: 53%

Pull #816

circleci

stop offering Herbie the retired Utah Pando mirrors

Herbie tries each source it knows until one has the file, so a dead mirror
costs nothing when the file exists and everything when it does not. The
University of Utah Pando archive is being wound down in favour of NOAA's
Registry of Open Data on AWS, and pando-rgw01/02.chpc.utah.edu now resolve in
DNS while accepting no connections.

check_hrrr_dataset_availability asks precisely the question that walks the
whole source list: is this time available? For an unavailable time Herbie
reached both dead hosts and blocked on each until TCP timeout, raising
ConnectTimeout after ~150s instead of returning False. Measured on the
2016-08-09 16:00 case the test uses: 151.9s and an exception before, 1.1s and
the correct answer after.

Filter the retired entries out of the source list the model's own Herbie
template declares, rather than hard-coding the survivors. New mirrors are then
picked up automatically, and each model keeps Herbie's ordering -- hrrr tries
AWS first while hrrrak tries NOMADS first, so a single hard-coded list would
silently reorder one of them. Model names are lower-cased first, as Herbie
does internally, because the GUNW workflow reaches this code with 'HRRR'.

Note this is not fixed by upgrading Herbie: current Herbie main still lists
both Pando entries, and no upstream issue tracks it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

31 of 31 new or added lines in 3 files covered. (100.0%)

3338 of 6488 relevant lines covered (51.45%)

0.51 hits per line