zentralopensource / zentral / 35738404796

91%

push

github

mdm: A throttled DEP assignment waits for the next synchronization

One device that Apple throttles made the task retry, and each retry was the
whole work list again. On the SQS broker that cost much more than a wasted
request. Celery keeps a countdown in the memory of a worker and leaves the
message unacknowledged, so the queue made the message visible again after its
visibility timeout, another worker took a copy, and each copy published a retry
of its own. Five levels of backoff turned one throttled device into tens of
thousands of messages, and the other tasks waited behind them.

The retry was never the only scheduler. The sync_dep_devices command is the
cron entry point, and it assigns the default enrollment inline, with no worker
and no retry of its own. The comments of the task already said that giving up
was not losing the work. The task stops now. The work list is read from the
database, so a throttled device keeps its place in it, and the next
synchronization finds it again. The result gives the number of throttled
devices, and the logs name each one.

The lock branch and the client errors stop the same way. A synchronization that
holds the lock schedules an assignment of its own when it commits, and a
request that Apple rejects or throttles waits for the next synchronization.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

5 of 5 new or added lines in 1 file covered. (100.0%)

52724 of 58009 relevant lines covered (90.89%)

0.91 hits per line