lfoppiano / grobid-quantities / 31593463157

39%

push

github

Request queue timeout  (#196)

Requests over maxParallelRequests are queued, not rejected; the queue was bounded by whatever the underlying Jetty version defaulted to.

Under Jetty 9's QoSFilter - the version that produced the behaviour in the report - an over-limit request was suspended with the servlet default async timeout of 30 seconds, then rejected with 503. That is where the reported '503 after ~30s despite idleTimeout: 120 seconds' came from: idleTimeout is a connection setting and never had any
bearing on it.

Jetty 12's QoSHandler, in use since the Dropwizard 5 migration, defaults differently again: a 1024-long queue and no waiting limit. 
Rather than swap one set of implicit defaults for another, set all three bounds from the configuration:

  maxQueuedRequests             (1024, negative = unbounded)
  maxQueuedRequestTimeout       (0 seconds = wait indefinitely)
  maxQueuedRequestsRejectStatus (503)

and log the effective values at startup, so a 503 can be traced back to a written-down value.

Neither these nor idleTimeout bound how long a request may take once it holds a slot; restAPI.md now says so explicitly.
So the 30s cut-off the issue reported is gone, and the connection idle timeout
no longer bounds the response computation in either direction - which is what
`idleTimeout` is supposed to mean. The 503 the reporter also saw under load
comes from `maxParallelRequests`, a different knob.

Documents the distinction, since the two were conflated throughout the thread.

* config: name the queue settings requestQueue*, not maxQueuedRequests*

Dropwizard already has a server.maxQueuedRequests, bounding the Jetty thread pool's queue, and doc/gettingStarted.md mentions it in the 0.8.0 upgrade notes. Introducing a top-level maxQueuedRequests next to it - in the branch fixing an issue that is *about* two similarly named
settings being conflated - would have been asking for it.

  maxQueuedRequests            -> requestQueueMaxSize
  maxQueuedR... (continued)

1004 of 2984 branches covered (33.65%)

Branch coverage included in aggregate %.

2638 of 6434 relevant lines covered (41.0%)

1.64 hits per line