Azure / ARO-RP / 33741291236

41%

Build Type

push

github

Committed by web-flow

Commit Message

[ARO-29322] Bound MIMO manifest retries when the cluster document cannot be read (#5060)

* fix(mimo): bound manifest retries when the cluster document cannot be read

The actuator fetched a fresh cluster document before leasing the manifest it
was about to action, and returned outright if that fetch failed. Two
consequences followed, both of which the fetch failing durably makes
permanent.

Lease is what increments Dequeues, so a manifest whose cluster document
could not be read never had its attempt count raised. maxDequeueCount was
therefore unreachable on this path and the manifest was retried on every
pass, indefinitely, rather than eventually being marked RetriesExceeded.

The early return also abandoned every remaining manifest queued against that
cluster, unlike the adjacent error branches, which log and continue. Being
unable to read the cluster document says nothing about the other manifests
in the batch, and they were not attempted at all.

Fetch after leasing, so the attempt is counted, and end the lease rather
than returning. The error is marked transient, so that a momentary read
failure is retried rather than failing the manifest outright, while a
durable one is now bounded by maxDequeueCount as any other repeated failure
is.

Extract the existing state mapping into manifestStateForError so that both
this path and task execution share it, rather than restating the dequeue
bound. Log output is unchanged.

* docs(mimo): note the cost of the RetriesExceeded terminal state

Record that a manifest given up on misses its scheduled window rather than
being deferred, and that the recurrence of schedules is what keeps this
bounded.

* fix(mimo): keep the original wording of the fetch-failure log

The wrapped TransientError was being logged in place of the original
message, changing the text queries and alerting match on. Log before
wrapping, and correct a typo in the test's field name.

Coverage Stats

35 of 37 new or added lines in 1 file covered. (94.59%)

7 existing lines in 2 files now uncovered.

32117 of 78746 relevant lines covered (40.79%)

0.47 hits per line