rero / rero-ils / 24720163436

91%

push

github

feat(documents): add async cover image support with Redis queue

- Add cover property to Document returning (url, provider) from electronicLocator
- Add AddCoverUrlExtension with pre_create/pre_commit hooks that enqueue
  documents with ISBNs into a Redis-backed set queue (deduplicating rapid saves);
  cover URL lookup is encapsulated as an instance method
- Add Redis queue worker (process_cover_url_queue) that serialises thumbnail
  lookups one at a time with a configurable delay to avoid race conditions and
  external-service overload
- Add pause/resume support: worker does not start while queue is paused,
  allowing setup to accumulate PIDs without concurrent ES version conflicts
- Add batch task add_cover_urls for CLI-driven bulk processing with per-provider
  count reporting and error handling
- Add CLI commands: add-cover-urls, cover-url-queue-status,
  cover-url-queue-worker (pause/resume), cover-url-queue-clear
- Sync RERO_INVENIO_THUMBNAILS_URL with RERO_ILS_URL via finalize_app so env
  overrides are respected
- Add config keys: RERO_ILS_COVER_URL_TASK_DELAY, RERO_ILS_COVER_URL_QUEUE_KEY,
  RERO_ILS_COVER_URL_WORKER_KEY, RERO_ILS_COVER_URL_PAUSED_KEY
- Move document-specific CLI commands into a new cli/documents.py module under
  the `documents` group: validate-documents-with-items-lofis,
  create-documents-with-items-lofis, marc21tojson, add-cover-urls,
  cover-url-queue-{status,worker,clear}; update scripts/setup accordingly
- Remove obsolete --create_items_holdings_small (-s) and
  --create_items_holdings_big (-b) options from setup script: the equivalent
  coverage is provided by --deployment for big fixtures; the conditional
  items/holdings generation blocks and their mutual-exclusion guards are gone

Co-Authored-by: Peter Weber <peter.weber@rero.ch>

371 of 624 new or added lines in 7 files covered. (59.46%)

24005 of 26346 relevant lines covered (91.11%)

0.91 hits per line