decentraland / atlas-server / 39821345489

94%

Build Type

push

github

Committed by web-flow

Commit Message

fix: read complete and consistent snapshots from the subgraph (#153)

Stacked on #152 (`fix/map-pipeline-state`). Review and merge that one
first.

## What was wrong
- **Large estates**: nested `estate { parcels }` lists default to
`first: 100`. Estate 2092 has 9,607 parcels, but an estate-level change
(name, owner, order) only reached 100 of them, and estate proximity came
from those same 100.
- **Truncated increments**: the update queries stopped at `first:
API_BATCH_SIZE`, and the cursor moved to the last row's `updatedAt`, so
rows that shared that second and didn't fit were skipped.
- **No snapshot**: the full fetch paged for minutes against a moving
head, and its cursor was the newest row it saw, so changes to
already-read pages could be lost.
- **Rentals boot**: thousands of `nftIds`-batched URLs.

## What changed
- Every fetch, full and incremental, reads `_meta { block { number
timestamp } }` and pins all of its queries to `block: { number }`. The
land cursor is that block's timestamp. I confirmed `_meta`, `block:`
pinning and `searchParcelEstateId_in` on
subgraph.decentraland.org/marketplace.
- Changed parcels and estates are paged by `id_gt` under a fixed
`updatedAt_gt` filter until a short page, so no rows are lost at
boundaries. All parcels of every affected estate (updated estates, plus
the ones updated parcels joined or left) are fetched the same way.
Estate NFTs and their proximity are built from the full parcel list.
- Rentals boot uses a paginated `status=open` sweep (82 listings in prd
today, 1 request). The same sweep runs again every
`RENTALS_SWEEP_INTERVAL` (default 3600 s) and fixes tiles whose listing
disagrees with it. Open listings that have already expired are skipped.

## Verification
`tsc --noEmit`, the full jest suite (126 tests) and prettier 2.8.8 all
pass. I checked that the new tests fail when each fix is reverted. I
also ran the component against the live subgraph with a 30-day cursor: 5
queries, 3.6 s, 1,398 tiles, an... (continued)

Coverage Stats

1222 of 1357 branches covered (90.05%)

Branch coverage included in aggregate %.

527 of 547 new or added lines in 7 files covered. (96.34%)

76 existing lines in 5 files now uncovered.

4327 of 4650 relevant lines covered (93.05%)

1133.36 hits per line