esnet-security / SCRAM / 20375315163 / 1

94%

develop: 94%

push

github

fix(process_updates): Fix expirations, cleanup syncing, and add integration tests (#157)

This one was a doozie. Expired/deleted entries on one SCRAM instance
were never unblocked on other instances because there is no mechanism
for postgres to let django know that it needs to re-run stuff, only the
active django re-runs stuff. Now, we get around that by calling
process_updates and re-updating anything changed recently, however,
before, the sync logic only looked at creation time (`when` field),
missing any entries that were removed..

To fix this in a lot less dumb way than we used to, we now we use
`django-simple-history` to detect _**any**_ entry changes and then go
ahead and just reprocess them to what the DB says they should be doing.

There is a lot in this PR, but the main changes are:


- Split up `process_updates()` into:

  - `get_entries_to_process(cutoff_time)`
- This is a function that we could reuse that simply finds all recently
modified entries from other instances and returns them so you can Do
Stuff® on em.
  - `reprocess_entries(entries)`
- This actually "Does Stuff" by sending websocket messages to
translators. We could eventually move this out of here and make it more
generic to be used everywhere we Do A Websocket® but it needs to be a
bit more dynamic to support future action types (see note below, not
dealing with this now).
  - `_check_for_orphaned_history()`
- This is a total edge case, but since we don't ever delete entries (our
delete() override) I wanted to make sure that we catch and log any
orphaned history objects that don't have a corresponding entry (say
someone goes into the admin panel and actually "hard" deletes an entry,
but the history stuff is still there).

- Made sure to be efficient with DB calls by using
`select_related()`,`.exclude()`, and `values_list()` on most of our
django/db queries to make sure that we're relying on SQL to do the
filtering (using built in django stuff obviously) instead of iterating
o... (continued)

64 of 71 branches covered (90.14%)

Branch coverage included in aggregate %.

1282 of 1368 relevant lines covered (93.71%)

0.94 hits per line