grobidOrg / grobid / 24021807914

push

github

Crossref consolidation revision  (#1398)

CrossRef request handling refactor:
- Removed duplications and fixed concurrency issues in request handling
- Initialized connection pool based on CrossRef tier (Plus/Polite/Public)
- Added exponential backoff with jitter for retries
- Replaced fixed 50-concurrent-request start with a ramp-up, dynamically modulating the pool from CrossRef response concurrency headers

Token validation:
- Validate CrossRef API token at startup to set concurrency limits
- On token validation failure, disable the token and adjust concurrency accordingly

Config & docs:
- Updated configuration
- Documented new CrossRef rate-limiting config options
- Added FAQ on GROBID consolidation's impact on PDF processing speed
- Added performance notes for CrossRef Plus and Polite tiers

8231 of 23987 branches covered (34.31%)

Branch coverage included in aggregate %.

17401 of 42919 relevant lines covered (40.54%)

6.63 hits per line