kobotoolbox / kpi / 17841054955

81%

master: 76%

Build Type

push

github

Committed by web-flow

Commit Message

feat(projectOwnership): store all transfer errors  (#6268)

### 📣 Summary
Retain all errors from transferring a project, displaying up to 100 in
Django admin.

### 📖 Description
When transferring a project, there may be any number of errors that
occur when handling individual files. Store individual errors for
individual files instead of just the broad 'Some attachments could not
be transferred' for easier debugging. These individual errors will be
displayed in Django admin.

### 👷 Description for instance maintainers
Adds a trivial migration (new table). Table may become very large if
there are lots of transfers with lots of errors.

### 💭 Notes
Creates a new model for storing all errors from a transfer since if we
are transferring a project with a lot of attachments and they all fail
we could end up with thousands of errors. We do want to track them all,
but not in one long string field, so we add the TransferStatusError
object. If there are more than a hundred errors, we don't want to
overwhelm the display in Django admin, so we just show the first 100 and
it will be up to a developer to look for the rest.

Additional updates:

Make the task to move media files more like the task to move attachments
by recording individual errors.

Updates the process to move files to reraise errors if requested instead
of just logging and swallowing them. This allows us to have more
specific error logs rather than just "Failed to move attachment."

Updates the save method on attachments to only update the user_id if
`update_fields=['user_id']` rather than going through all the
recalculation of hash and size. This prevents us from having duplicate
errors from trying to move and then save a busted file.

Record an error if we're trying to update the status of a TransferStatus
from 'success' to something else. This should never happen but we have
reason to believe there is a race condition that is making it happen.

### 👀 Preview steps
Make sure to restart the worker... (continued)

Run Details

6485 of 10179 branches covered (63.71%)

21 of 60 new or added lines in 5 files covered. (35.0%)

2 existing lines in 2 files now uncovered.

24530 of 30124 relevant lines covered (81.43%)

1.6 hits per line