kubernetes-sigs / lws / 41309524015

88%

Build Type

push

github

Committed by web-flow

Commit Message

Charge the group restart budget once per group failure (#1249)

* Charge the group restart budget once per group failure

A group failure often restarts several of its pods at once. The first
pod event charges the budget and deletes the leader. The informer cache
can then still show that leader alive while it already shows the new
count, so the next pod event charges the budget again. With
maxGroupRestarts=3, one failure that restarts several workers can use
the whole budget.

Read the leader from the API server before charging the budget, and
stop when it is already deleting, gone, or replaced.

* Deduplicate group restart charges by leader UID

Persist the charged leader UID atomically with restart counts instead of
reading the leader directly from the API server. Retry deletion without
charging or applying backoff again, and prune UIDs with their counters.

Cover stale Pod caches, conflict retries, deletion failures, replacement
leaders, and restart-budget cleanup paths.

Signed-off-by: Max Yang <pandacooming@gmail.com>

* Cover charged UID cleanup during Hash group handoff

Adapt the Hash restart-count handoff to the combined count and UID
mutation and verify that the replacement inherits the count without
being marked as already charged.

Signed-off-by: Max Yang <pandacooming@gmail.com>

* Adapt Hash scale-down pruning to charged UID bookkeeping

Update the remaining restart-count mutation callback and exercise UID
cleanup when unclaimed Hash counters are pruned after scale-down.

Signed-off-by: Max Yang <pandacooming@gmail.com>

* Mark restart budget test pods as running

ContainerRestarted only reports restarts for Running or Pending Pods, so
the new restart budget unit tests returned before reaching the charge
logic. Set the Pod phase so the tests exercise the deduplication paths.

Signed-off-by: Max Yang <pandacooming@gmail.com>
Claude-Session: https://claude.ai/code/session_01XQVyjtCCPZtvSY7WbqRW7R

* Bound charged leader history and verify ... (continued)

Coverage Stats

65 of 75 new or added lines in 3 files covered. (86.67%)

2 existing lines in 1 file now uncovered.

9357 of 10591 relevant lines covered (88.35%)

36628.81 hits per line