pomerium / pomerium / 40401392926

54%

Build Type

push

github

Committed by web-flow

Commit Message

fix: xdsmgr late ack (#6900)

## Summary

xdsmgr streams config to Envoy over delta ADS. For each resource type it
keeps a record of the resource versions it believes Envoy holds, and on
every config change it sends only the resources whose version differs
from that record. Versions are content hashes, so a resource that
returns to earlier content gets its earlier version back.

Before this change the record was updated only when Envoy ACKed the
**latest** nonce, and an ACK of any earlier response was dropped. Envoy,
however, applies every response on the stream in order before it
acknowledges it, and may acknowledge late: it holds the CDS ACK back
while a cluster warms. If the config went A → B → A before the ACK of B
arrived, the record still said A, the return to A diffed empty, nothing
was sent, and Envoy kept B for the life of the stream. The affected
route answered 404 `route_not_found` (or 503 `cluster_not_found` when
only the CDS ACK was late) until Pomerium restarted, while xDS health
reported running and later config rebuilds that contained the route sent
nothing.

```mermaid
sequenceDiagram
    autonumber
    participant CS as Config source
    participant X as xdsmgr
    participant E as Envoy
    Note over X,E: Before. Envoy has A (route and cluster flip). xdsmgr's record says A.
    CS->>X: update to B, route removed
    X->>E: route config main, remove cluster flip, nonce n
    E->>E: apply B, route and cluster gone
    CS->>X: update back to A, latest nonce n+1
    Note over X: diff of A against the record (still A) is empty, nothing sent
    E->>X: ACK nonce n
    Note over X: n is not the latest nonce, ACK dropped, record stays A
    Note over X,E: Envoy has B, xdsmgr believes A. Nothing is ever re-sent.
```

```mermaid
sequenceDiagram
    autonumber
    participant CS as Config source
    participant X as xdsmgr
    participant E as Envoy
    Note over X,E: After. Envoy has A. xdsmgr's record says A.
    CS->>X: update to B, route r... (continued)

Coverage Stats

60 of 67 new or added lines in 2 files covered. (89.55%)

23 existing lines in 8 files now uncovered.

35368 of 65591 relevant lines covered (53.92%)

331.08 hits per line