Freegle / Iznik / 24558

72%

Build Type

push

circleci

Committed by invalid-email-address

Commit Message

feat(digest): shard the daily digest (hash key) so it covers the userbase in-window

The daily digest ran a single un-sharded worker gated to a 5-hour 07:00-12:00 window.
At full cutover (~78k daily members) one worker reaches only a fraction before the window
closes, so most members backlog for days. Shard it like immediate/reach.

- console.php: 4 parallel daily shards (everyThirtyMinutes, 07:00-12:00). Kept at 4, not 8:
  this 6-core host's per-user cost (spatial reach-gate + MJML render) makes the bottleneck
  largely shared/downstream, so more workers add contention/CPU rather than throughput —
  8 saturated the box.
- Shard key is CRC32(users.id) % shards, NOT MOD(users.id, shards): under Galera/Percona the
  auto-increment stride equals the cluster size (auto_increment_increment) with a per-node
  offset, so raw ids are non-contiguous and a MOD split skews hard when the shard count
  shares a factor with the cluster size. A hash is uniform for any count and immune to the
  stride / cluster-size changes.
- SendUnifiedDigestCommand::lockKeySuffix() now keys the per-shard flock by MODE too, so
  daily shard N and immediate shard N no longer share a lock file and block each other.

Builds on the DIGEST_LOAD_CAP memory bound (prior commit) which keeps each shard's per-user
load from OOMing regardless of backlog depth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015DQRL5iGD71jNTgyokMpt9

Coverage Stats

12266 of 16222 branches covered (75.61%)

Branch coverage included in aggregate %.

0 of 4 new or added lines in 2 files covered. (0.0%)

30 existing lines in 2 files now uncovered.

130644 of 183122 relevant lines covered (71.34%)

37.9 hits per line