decentraland / deployments-to-sqs / 31953723569

90%

main: 91%

Pull #538

github

feat: record processed deployments in postgres

Replaces "an object exists in the bucket" as the record of what has been
processed, which conflated "downloaded" with "published" and silently
dropped entities.

If an SNS publish failed, markAsDeployed was correctly skipped so the entity
would retry — but the download had already written the object, so on retry
exist() said "already stored" and the entity was marked deployed without
ever being published. Reproduced against a real catalyst with a dummy SNS
ARN: 1,648 entities stored, 0 published, and on restart 1,657 skipped as
already-stored. An SNS outage did not delay deployments, it lost them.

A row is now claimed before the download and stamped with published_at only
after both publishes succeed. Dedup reads:

  row present  -> processed only if published_at is set
  no row       -> fall back to storage.exist()

The fallback is a migration step. Every entity processed since 2022 exists
only as an S3 object, and a table-only check would treat the whole bucket as
unprocessed and replay years of deployments at the converter. Claiming
BEFORE the download is what makes "no row" mean "predates the table" rather
than "publish failed" — without it the fallback would reintroduce the bug.

filterProcessedSnapshotsFrom now resolves a batch in one query instead of up
to 1000 sequential S3 HEADs, falling back to storage only for hashes the
table does not know.

published_at IS NULL now identifies stranded entities; the count is logged
at startup. Note it only covers entities seen since this ships — ones
stranded historically have an object and no row, and are indistinguishable
from never-seen ones.

Postgres runs on the shared supra cluster already used by 34 services. The
database and user still need provisioning, and
PG_COMPONENT_PSQL_CONNECTION_STRING must be set in definitions, before this
is deployed.

64 of 79 branches covered (81.01%)

Branch coverage included in aggregate %.

33 of 51 new or added lines in 5 files covered. (64.71%)

307 of 332 relevant lines covered (92.47%)

8.41 hits per line