iotaledger / iota / 37557297359

72%

Build Type

push

github

Committed by web-flow

Commit Message

fix(consensus): serve every peer while others stall or hoard admission slots (#13049)

# Description of change

Admission is charged when a request's headers land and released when its
response has been fully written. In between, a peer could keep a slot
busy at almost no cost, in two ways. It could send the request headers
and never the message; only the 120 s fallback `request_timeout` ended
that. Or it could send a valid request and never read the response; no
timeout ended that at all, because the transport polls a response body
only when the receiver grants window, and the server cannot reset one
stream. Commit fetches also draw on a budget of 16 shared by all peers,
so two peers at their per-peer cap of 8 held all of it and no other peer
was served.

Three changes:

- A request whose message has not fully arrived within 10 s of its
headers is answered with `DEADLINE_EXCEEDED`. The wrapper sits inside
the routes, around the generated service, because that is where the
request body carries tonic's type; the router layers see axum's.
- A peer holding no commit fetch is always granted one; further fetches
from a peer are granted only while the total held stays below
`max_commit_fetches_total`. The total is therefore exceeded by at most
one slot per peer, and a peer whose extra slots are cut cannot take them
back while other peers are being served.
- A response not fully written within the longest wait any requester has
for one fetch attempt, 480 s, gives up its contents and the node-wide
commit-fetch slot. The peer's own slot stays charged until the stream
really ends, so a dead stream counts against the peer that left it and
eight of them stop its commit fetches altogether. Frames handed to the
transport are 64 KiB copies, so what a reclaim cannot free stays small.
The block-subscription stream is exempt from both deadlines.

Both deadlines are constants. The request message is at most 1 MiB by
default, so 10 s covers any working link. The response... (continued)

Coverage Stats

639 of 675 new or added lines in 9 files covered. (94.67%)

587 existing lines in 28 files now uncovered.

186206 of 264254 relevant lines covered (70.46%)

441509.38 hits per line