kobotoolbox / kpi / 37768324112

83%

master: 76%

Build Type

push

github

Committed by web-flow

Commit Message

fix(trashBin): throttle attachment auto-deletion and stop locking the Celery Beat schedule row DEV-2999 (#7663)

### 📣 Summary
This PR stops trash bin operations from locking Celery Beat's schedule
row, makes attachment auto-deletion run in small batches, and keeps
storage counters and trash clean-up correct under load.

### 👷 Description for instance maintainers

New optional settings (environment variables):
- `AUTO_DELETE_ATTACHMENTS_MAX_PER_USER` (default `100`)
- `AUTO_DELETE_ATTACHMENTS_USERS_PER_RUN` (default `5`)
Each trashed attachment keeps its own Beat task until it is
hard-deleted, so raising these values grows the number of tasks Celery
Beat has to scan.
Migration `logger.0055` adds a partial index on `logger_attachment`.
When `SKIP_HEAVY_MIGRATIONS` is set, create it manually with the SQL the
migration prints.
Trash bin tasks may now start up to 5 minutes after their due time.

### 💭 Notes
1. **No Beat row update from the trash bin** (`utils/signals.py`):
`temporarily_disconnect_signals()` no longer calls
`PeriodicTasks.update_changed()`, which locked Beat's single
`PeriodicTasks` row until the caller's transaction committed, so
concurrent callers waited on each other. It is not needed:
django-celery-beat 2.9.0 reloads its schedule every 5 minutes and only
loads clocked tasks due within 5 minutes. Covers `move_to_trash()`,
`put_back()` and `garbage_collector()`.
2. **`auto_delete_excess_attachments`**:
- returns early when `AUTO_DELETE_ATTACHMENTS` is off, so tasks already
queued do nothing;
- the per-user lock now covers the work (it was released before the work
started);
- trashes at most `AUTO_DELETE_ATTACHMENTS_MAX_PER_USER` attachments per
run, oldest first, and the next run continues;
   - no more queries per attachment (3 → 0).
3. **`schedule_auto_attachment_cleanup_for_users`**: queues at most
`AUTO_DELETE_ATTACHMENTS_USERS_PER_RUN` users per run. Users take turns
through a cursor on `ExceededLimitCounter.id` kept in the cache.
4... (continued)

Coverage Stats

10334 of 13680 branches covered (75.54%)

27 of 136 new or added lines in 9 files covered. (19.85%)

5136 existing lines in 154 files now uncovered.

33042 of 39812 relevant lines covered (83.0%)

5.49 hits per line