agentjido / req_llm / 87679f7ddf3bee2963e2045d91c857908208349c

49%

push

github

Add OAuth2 token caching for Google Vertex AI (#174)

OAuth2 token generation on every request adds 60-180ms overhead
(file I/O, JWT signing, HTTP round trip). Tokens are valid for 1 hour,
so this overhead is wasteful for repeated requests.

Implementation:
- GenServer + ETS cache storing tokens per service account
- 55 minute TTL (5 minute safety margin before 1 hour expiry)
- GenServer serializes concurrent refresh requests to prevent duplicates
- Per-node cache (no distributed coordination needed)

Performance impact:
- Before: 1000 requests = 60-180 seconds auth overhead
- After: 1000 requests = 60-180ms auth overhead (first request only)
- Improvement: 99.9% reduction in auth overhead

15 of 26 new or added lines in 2 files covered. (57.69%)

3620 of 7366 relevant lines covered (49.14%)

56.92 hits per line