NVIDIA / nodewright / 31845152482

79%

Build Type

push

github

Committed by web-flow

Commit Message

fix(operator): release the webhook bootstrap lease when it cannot be used (#472)

`helm rollback` across the resource-name rename left the release `failed` with no
Ready operator pod, and did not self-heal (#469). The pod being rolled away holds
`nodewright-webhook-bootstrap.nvidia.com`, sees no webhook configuration dialling
its Service, so it never goes Ready; a rolling update will not terminate an
un-Ready pod, so it keeps renewing the lease; and the pod being rolled back to
cannot inject a caBundle without that lease, so it cannot go Ready either.

This is not specific to the rename. Ownership of a webhook configuration is "it
dials the Service I was told to serve", so any change to the webhook Service name
reaches the same state, and `webhook.serviceName` and `fullnameOverride` are
ordinary chart values. The forward direction is broken on those knobs today too:
`selectorMigration` does not fire, because the selector is unchanged and
WEBHOOK_SERVICE_NAME is present.

An operator that owns no webhook configuration now checks whether some other
Service is dialled in its namespace. If so the release has moved on without it,
and after a grace period it stops the bootstrap manager, which drops the lease,
and a fresh one contends again 30s later. Nothing is deleted, and the reconcile
manager, webhook server, and readiness are untouched; if no other pod takes the
lease, this one takes it back. The grace period keeps a healthy upgrade, which
applies the Deployment and the webhook configurations in one pass, from tripping
it. Detection lists without the label filter that ownership uses, because the
release being rolled back to may predate that label.

The serving certificate is the other half. Secret/webhook-cert is operator-owned
and survives the rollback, and the operator on the far side has no
remint-on-Service-change check, so the certificate it inherits has to already be
valid for the name it will serve. Minted certificates now carry a SAN for the
pre... (continued)

Coverage Stats

109 of 215 new or added lines in 3 files covered. (50.7%)

3 existing lines in 3 files now uncovered.

10323 of 13077 relevant lines covered (78.94%)

7.84 hits per line