stacklok / toolhive / 30343825881

70%

Build Type

push

github

Committed by web-flow

Commit Message

Fix workload status poisoning and deflake Envoy AllowPort (#6076)

ListWorkloads snapshotted the runtime before the status files. thv run
detaches before its container exists and the child flips the file to running
shortly after, so a list could observe "file running, runtime missing" and
persist unhealthy for a healthy workload -- with no recovery path, since
validateWorkloadInList skips revalidation for non-running file statuses. One
racy thv list permanently bricked a workload's status.

Read the files first: a file saying running at T implies the container existed
before T, so a later runtime miss is genuine. Guard the write as well --
reading files first newly allows a list that brackets an entire thv rm to
SetWorkloadStatus on a deleted file, and that call creates missing files,
resurrecting the workload as a ghost entry. The guard re-checks under the file
lock and skips if the file is gone or no longer says running/stopped.

This removes one trigger; the list path still writes on read, so the root
cause in #4432 stands.

Also retry the Envoy AllowPort spec's allowed-port leg and surface the fetch
error text. Its contract is steady-state reachability; success on the first
request after boot is not something Envoy's dynamic_forward_proxy path
promises. The blocked leg is untouched.

Refs #4432, #6078.

Coverage Stats

20 of 26 new or added lines in 1 file covered. (76.92%)

30 existing lines in 5 files now uncovered.

81290 of 116507 relevant lines covered (69.77%)

97.65 hits per line